Introduction to the AV Foundation

1.1 Definition of AV Foundation

AV Foundation is a high-level Object-C framework for handling time-based media data in APPLE OS X and iOS. Providing a vast set of features through the development of the required tools allows developers to create the most advanced media applications available on Apple’s platforms.

1.2 Scope of application of AV Foundation

One of the first steps to learning about the AV Foundation is to know where it fits into all of Apple’s media resources categories.

Framework and Functions
  • Core Audio

Core Audio is a framework for handling all Audio events on OS X and IOS. Core Audio is an umbrella term for how high frameworks are integrated, providing interfaces for recording, playing, and processing Audio and MIDI content. Core Audio also provides advanced interfaces, such as those provided through the Audio Queue Services framework, that handle basic Audio playback and recordrelated functions. There are also relatively low-level interfaces, particularly the Audio Units interface, which provides complete control over the Audio signal and allows you to build complex Audio processing patterns, Much like apple’s Logic Pro X and Avid’s Pro Tolls tools.

  • Core Video

Core Video is a pipeline mode for digital Video on OS X and IOS. Core Video provides image caching and caching support for its relative Core Media, providing an interface for frame-by-frame access to digital Video. The framework simplifies the complexity of converting between pixel formats and managing synchronization items.

  • Core Media

Core Media is part of the lower-level Media pipeline used by the AV Foundation. It provides low-level data types and interfaces for audio samples and video frame processing. Core Media also provides a time model based on the CMTime data type for the AV Foundation. CMTime and its related data types are typically used when the AV Foundation handles time-based operations.

  • Core Animation

Core Animation is a compositing and Animation framework provided by OS X and iOS. The main function is to provide the beautiful, smooth animation effects of the Apple platform. Provides a simple, line-declaring programming pattern, and has encapsulated various object-C based classes that support OpenGL and OpenGL ES functionality. When using Core Animation, AVFoundation provides hardware acceleration mechanism to optimize the whole process for the two actions of playing food content and capturing video. AVFoundation also leverages Core Animation to enable developers to add Animation titles and image effects during video editing and playback.

In between the high-level and low-level frameworks is the AV Foundation. AV Foundation plays an important role in the overall media classification diagram, as it provides many functions and performance that can only be realized by low-level frameworks, and is implemented in a simpler object-C interface. It also works seamlessly with higher-level frameworks, such as Media Player and Assets Library, using the services provided by these frameworks and interacting directly with Core Media and Core Audio when more advanced features are required. In addition, since AV Foundation is under the UIkit and AppKit layers, it also means that you can use the media framework on both platforms.

Parsing the AV Foundation.

Some of the core functions provided by the AV Foundation
1.3.1 Audio and video playback and recording

AVAudioPLayer and AVAudioRecoder provide an easier way to integrate audio and video playback and recording in your application. These methods aren’t the only ways the AV Foundation uses to play and record, but they are the easiest and most powerful to learn.

1.3.2 Video Playback

The AV Foundation framework allows you to play video resources from local files or remote streams and control video playback and content presentation. The core classes are AVPlayer and AVPlayerItem.

1.3.3 Media Capture

AV Foundation provides a set of APIS for precise control of the device. The core class for camera capture is AVCaptureSession, which serves as a gathering point for all activities to receive movies and pictures sent by various streams of the camera device.

1.3.4 Media Editing

The AV Foundation provides strong support for the integration and editing of media resources. Allows you to create applications that combine multiple audio and video resources, modify and edit individual media clips, modify the parameters of audio files, and add animation titles and scene switching effects.

1.3.5 Media Processing

When you need to perform more advanced media processing tasks, you can use the AVAssetReader and AVAssetWriter classes to do this. They provide direct access to video frames and audio samples.