In iOS 4.0 Apple introduced the AV Foundation APIs that made working with audio and video media much easier than it had been in previous versions of iOS. Apple then brought these APIs to Mac OS X in OS X 10.7 "Lion". In this post I’ll show how to use some of the APIs to create a simple four track mixer.
The code for the project is available on GitHub. You’ll need to provide your own audio files. The project is setup to use four WAV files named track1.wav – track4.wav but the ReadMe.md lists the two lines of code you need to change to use another audio format or naming scheme. Obviously you need to use an audio format supported by iOS. The project is configured to use ARC (Automatic Reference Counting).
Setup
To start we’ll create an AVMutableComposition instance and two NSMutableDictionaries. We’ll use the dictionaries for keeping track of the IDs assigned to the audio tracks by the AVFoundation framework and for storing the volume level of each track.
Our audio tracks are loaded from the Resources directory within the app bundle using the AVURLAsset class and then used to create an AVMutableCompositionTrack that we insert into our AVMutableComposition. We record the ID of the track and set the track volume to 1.0, the maximum. The volume for each track can be set from 0.0 – 1.0.
// Setup
_composition = [AVMutableComposition composition];
_audioMixValues = [[NSMutableDictionary alloc] initWithCapacity:0];
_audioMixTrackIDs = [[NSMutableDictionary alloc] initWithCapacity:0];
// Insert the audio tracks into our composition
NSArray* tracks = [NSArray arrayWithObjects:@"track1", @"track2", @"track3", @"track4", nil];
NSString* audioFileType = @"wav";
for (NSString* trackName in tracks)
{
AVURLAsset* audioAsset = [[AVURLAsset alloc]initWithURL:[NSURL fileURLWithPath:[[NSBundle mainBundle] pathForResource:trackName ofType:audioFileType]]
options:nil];
AVMutableCompositionTrack* audioTrack = [_composition addMutableTrackWithMediaType:AVMediaTypeAudio
preferredTrackID:kCMPersistentTrackID_Invalid];
NSError* error;
[audioTrack insertTimeRange:CMTimeRangeMake(kCMTimeZero, audioAsset.duration)
ofTrack:[[audioAsset tracksWithMediaType:AVMediaTypeAudio]objectAtIndex:0]
atTime:kCMTimeZero
error:&error];
if (error)
{
NSLog(@"%@", [error localizedDescription]);
}
// Store the track IDs as track name -> track ID
[_audioMixTrackIDs setValue:[NSNumber numberWithInteger:audioTrack.trackID]
forKey:trackName];
// Set the volume to 1.0 (max) for the track
[self setVolume:1.0f forTrack:trackName];
}
// Create a player for our composition of audio tracks. We observe the status so
// we know when the player is ready to play
AVPlayerItem* playerItem = [[AVPlayerItem alloc] initWithAsset:[_composition copy]];
[playerItem addObserver:self
forKeyPath:@"status"
options:0
context:NULL];
_player = [[AVPlayer alloc] initWithPlayerItem:playerItem];
Finally, we create an AVPlayerItem using our AVMutableComposition, and use that to create a AVPlayer that we’ll use to control playback of the composition. We also want to observe the "status" key of our player item to be notified of changes in the state of our AVPlayerItem.
Key Value Observing (KVO)
- (void)observeValueForKeyPath:(NSString *)keyPath ofObject:(id)object change:(NSDictionary *)change context:(void *)context
{
if ([keyPath isEqualToString:@"status"])
{
if (AVPlayerItemStatusReadyToPlay == _player.currentItem.status)
{
[_player play];
}
}
}
This code is pretty straightforward. If our AVPlayerItem is ready to play, then go ahead and start playing the audio composition that we created above.
Slider Action
// Action for our 4 sliders
- (IBAction)mix:(id)sender
{
UISlider* slider = (UISlider*)sender;
[self setVolume:slider.value
forTrack:[NSString stringWithFormat:@"track%d", slider.tag]];
[self applyAudioMix];
}
Our UI has four UISliders, each of which call this action method. The sliders are tagged from 1 to 4 and we use the tag to map to the corresponding audio track. The value of the slider is configured to be between 0.0 and 1.0 so we use it to set the track volume and then apply the updated audio mix to the composition.
Setting the Volume
// Set the volumne (0.0 - 1.0) for the given track
- (void)setVolume:(float)volume forTrack:(NSString*)audioTrackName
{
[_audioMixValues setValue:[NSNumber numberWithFloat:volume] forKey:audioTrackName];
}
When we set the volume we simply update the value in our NSMutableDictionary that stores the volume level for each track.
Applying the Mix
// Build and apply an audio mix using our volume values
- (void)applyAudioMix
{
AVMutableAudioMix* mix = [AVMutableAudioMix audioMix];
NSMutableArray* inputParameters = [[NSMutableArray alloc] initWithCapacity:0];
[_audioMixTrackIDs enumerateKeysAndObjectsUsingBlock:^(id key, id obj, BOOL*stop) {
AVAssetTrack* track = [self trackWithId:(CMPersistentTrackID)[(NSNumber*)obj integerValue]];
AVMutableAudioMixInputParameters* params = [AVMutableAudioMixInputParameters audioMixInputParametersWithTrack:track];
[params setVolume:[[_audioMixValues valueForKey:key] floatValue]
atTime:kCMTimeZero];
[inputParameters addObject:params];
}];
mix.inputParameters = inputParameters;
_player.currentItem.audioMix = mix;
}
Here’s where we update and apply the audio mix to our composition. First we create an instance of the AVMutableAudioMix class and then for each track create an AVMutableAudioMixInputParameters instance for the track with the track volume as determined by the current slider value that we’ve stored. We add the AVMutableAudioMixInputParameters to an array and then set the parameters for our AVMutableAudioMix instance. Finally, we set the mix for the current item of our AVPlayer to our newly created mix.
Closing Thoughts
In just a few methods we have a simple four track mixer. This would have been much harder to accomplish without AVFoundation since we’d likely need to use the low-level Core Audio APIs.
It will be interesting to see what updates Apple has planned for AVFoundation in iOS 6.0 and OS X 10.8 and if the related QuickTime APIs in OS X will be deprecated in favor of AVFoundation.
