[CocosDenshion] CDSoundEngine and AudioQueue issue

cocos2d for iPhone » Programming » cocos2d support (graphics engine)

[CocosDenshion] CDSoundEngine and AudioQueue issue

(14 posts) (4 voices)

Started 2 years ago by PhilM
Latest reply from Steve Oldmeadow

Tags:

PhilM
Member

Hi,

I've recently got CocosDenshion (2.0 from recent svn, running on cocos2d 0.8 svn as of yesterday) integrated into my app using the CDSoundEngine to play multiple samples based on the drum pad demo, and thats been working great (thanks very much Steve!).

I'm now trying to add a recording feature using Audio Queue, based on the tutorials from http://trailsinthesand.com/exploring-iphone-audio-part-1/. I can get it working fine if I don't load the audio manager (commenting out [CDAudioManager initAsynchronously:...] from the app delegate).

But I can't get the recording to work on the handset with CocosDenshion loaded. Audio Queue init's fine, it just won't stay running (some issue with AudioQueueEnqueueBuffer it seems).

On the simulator its even stranger, as its the opposite way round, but I'm only really worried about the device right now.

Is it possible to have both sound engines/interfaces running at the same time? Or is it possible to totally unload CocosDenshion whilst I run the other (I've seen in other posts you say its not a good idea, but maybe I could live with the leaks if their small)

I feel I'm getting in way over my head now, after spending hours trying to debug this, and seemingly going round in circles :-/

Posted 2 years ago #
Steve Oldmeadow
Administrator

Theoretically you could have them co-exist but you will need to manage the audio session yourself so make sure you don't initialise CDSoundEngine with an audio session category otherwise it will manage the audio session. You definitely don't want to use CDAudioManager or SimpleAudioEngine as they both assume they will manage the audio session. Have you read Apple's guide to audio sessions? If not, you need to read it because recording audio introduces a bunch of new complications.

You could try interrupting CDSoundEngine before recording and then resuming it when the recording finishes but someone else was trying to do the same thing and I think they had no success doing that.

Have you looked at AVAudioRecorder? It is only available in 3.0 but I assume Apple introduced it due to the great amount of pain trying to use AudioQueues causes people.

CDSoundEngine is designed to be torn down and restarted but as you pointed out there are some leaks in the OpenAL code if you do this. It was about 3Kb from memory.

Posted 2 years ago #
PhilM
Member

Excellent tips & info Steve, thanks very much!

So I'd basically need to implement my own version of CDAudioManager to get it working. I hadn't totally grasped the separation between it & CDSoundEngine. I'll have a pick through the code to learn how to init and interrupt CDSoundEngine myself.

I was originally hoping to keep at least 2.2+ compatibility, mainly for ipod touch users that haven't paid for their upgrade yet, although I've just seen that you need to use a headset with them, so maybe thats not so important, as I doubt many ipod users have stumped up the extra for a pair :-/

Its only a minor addition to a small side project I'm hoping to submit to apple this week, so will leave the recording for a future update. I just wanted to check how feasible it was, looks like it'll need quite a bit of work to integrate properly.

I'll look into AVAudioRecorder next, and have a proper read of Apples audio docs (I've gone through about half of them so far, including a large chunk of the AudioQueues text, slowly digesting it all!).

I take it if I use AVAudioRecorder, I still need to avoid CDAudioManager & SimpleAudioEngine?

CDAudioManager handling the sessions has made my life at least 10x easier this week! I'd rather ship with audio playback working perfectly (which it seems to without the recording feature), & spend more time polishing the interface and code, before trying to learn more about audio on the handset and attempting this properly.

Thanks again for the help, and the amazing audio framework ; )

Posted 2 years ago #
Steve Oldmeadow
Administrator

The main problem with using CDAudioManager and recording is it doesn't have any audio session categories that allow recording. You also need to make sure the audio session is only initialised once which can be a problem when trying to combine several audio examples. I am planning to do something with recording audio but not for at least a couple of months.

Posted 2 years ago #
Steve Oldmeadow
Administrator

I came across this game that claims to be using CocosDenshion and allows recording of sound effects. Maybe you could e-mail them and ask them how they did it.

http://www.aftershockapp.com/index.php

I noticed it is only 3.x compatible as it has in app purchasing so maybe they are just using AVAudioRecorder. Looks like this is from the guy who asked me about recording audio previously (Ryan Barnard) so hopefully he will feedback what is required.

Posted 2 years ago #
PhilM
Member

I think I ran into the initializing one audio session restriction, when loading the AudioQueue code, it would stop the playback working (at least on the simulator).

I've just emailed the info email contact on Ryan's website link you gave. Hopefully he'll see it & has a minute to reply.

Either way, your help has been invaluable Steve, and given me some good info on what I need to look into next. Cheers!

I'll keep you posted on my progress in the upcoming weeks, once I've got this initial launch out of the way...

Posted 2 years ago #
RyanBarnard
Member

Hi Phil (and Steve),

I ended up using AVAudioRecorder along with CocosDenshion. I'm not at my computer right now that has the code showing how I did it, but tonight I'll come back and post my solution. It actually went very smoothly...the stupidest part of it was that I spent FOREVER trying to get it to work, only to find out that my 1st attempt at getting it to work was just fine, just the iPhone simulator wasn't working right.

The process to get it to work was:
1) Start up CocosDenshion.
2) Play sounds as the app progresses.
3) Pause CocosDenshion.
4) Start up AVAudioRecorder session.
5) Record audio & save to file.
6) Shutdown AVAudioRecorder.
7) Resume CocosDenshion
8) Load new audio file
9) Play sounds (new one w/ old one).

On step 7 the iPhone simulator was throwing timeout errors, but the actual iPhone worked just fine.

As I said, I'll come back tonight and post some code for how I did it. It's not the most elegant solution because I was close to finishing my app and was more interested in getting it finished than "properly" tying it into the code. But it still works just fine.

Steve, I have no clue how you found my app's website so fast! I just submitted the game to Apple on Friday night, so hopefully it'll get approved soon. I still have to get my final art set up on that site before it gets approved. The work never ends, does it? :)

Posted 2 years ago #
PhilM
Member

Hi Ryan,

Thanks for replying so quickly.

Interesting to know you opted for AVAudioRecorder. Very pleased to hear you got it all working, regardless of how 'elegant' your solution was, it sounds like you've made good progress!

I've been surprised how different the simulator reacts to audio compared to the handset, they seem to be handled totally different, with errors on one, not appearing on the other & vice versa... It took me hours to realize that!

Look forward to seeing any sample code/snippets you can provide, I really appreciate the help!

Good luck with your release and website updates. I look forward to trying out your game. It sounds like a more indepth variation of sneezies? (I've not played the facebook app you mention). Very interested to see your in-app purchase setup as well. Sounds like your making good use of the 3.0 features : )

I agree, my to-do list always seems to be growing, not decreasing! All this was supposed to be for a small side project that should of only taken one to two weeks of spare time. I'm now in week 2 of spending pretty much full time working on it! Can't wait to get back to my main game development (after unfortunately having to catch up with some regular paid work)...

Thanks again.

Posted 2 years ago #
Steve Oldmeadow
Administrator

@Ryan - thanks for the info, I found your site because it mentioned CocosDenshion.

Posted 2 years ago #
RyanBarnard
Member
Alright, I've got my code! :)

The first thing that I did was stop the CDAudioManager with a "[am audioSessionInterrupted]".

Second, I used the code from this forum post as my base code, adjusting it as required (which wasn't very much): http://stackoverflow.com/questions/1010343/how-do-i-record-audio-on-iphone-with-avaudiorecorder
(Thank you to whoever posted that!)

Lastly, to get the CDAudioManager back, I did:
```
[[AVAudioSession sharedInstance] setActive:NO error:nil];
[am audioSessionResumed];  // Resuming CDAudioManager
```
And then, of course, you probably want to load your new sound effect, so:
```
[soundEngine unloadBuffer:sfxId];
[soundEngine loadBuffer:sfxId fileName:pathToSound fileType:@"caf"];
```
Now, I made a couple of modifications to both the recording code that I listed above and also to soundEngine. First, here's the big change to the AVAudioRecorder:
```
NSArray *documentPaths = NSSearchPathForDirectoriesInDomains(NSDocumentDirectory, NSUserDomainMask, YES);
documentsDir = [[documentPaths objectAtIndex:0] retain];
NSString* fullFilePath = [documentsDir stringByAppendingPathComponent:@"RecordedAudio"];
NSURL *url = [NSURL fileURLWithPath:fullFilePath];
```
And that is the url that gets passed into the AVAudioRecorder initWithURL method. That puts the recorded file into the Documents directory of the app.

And here's the change that I made to CDSoundEngine's loadBuffer method...near the top of it when it's creating the fileURL:
```
NSString * path = [[NSBundle mainBundle] pathForResource:fileName ofType:fileType];
	if (path) {
		fileURL = (CFURLRef)[[NSURL fileURLWithPath:path] retain];
	}
	else
	{
		fileURL = (CFURLRef)[[NSURL fileURLWithPath:fileName] retain];
	}
```
Basically, it first checks the bundled resources for the audio file being referenced. If it's not found, then it checks the URL path that was passed in.

Oh, finally, to get all this code to work, I had to change my source files that uses AVAudioRecorder from ".m" to ".mm"

If you have any more questions, I'd be glad to assist. Hope this all helps!
Posted 2 years ago #
Steve Oldmeadow
Administrator

Thanks for the code Ryan. Thanks to AkumaStreak the 2.0 release of CocosDenshion changed all the file related stuff to use a path instead of a file name so I don't think the change to CDSoundEngine would be needed any more so it should be even easier to implement.

Posted 2 years ago #
PhilM
Member

excellent post Ryan, thanks soooo much!!!

This will surely save me (& hopefully others) alot of time.

I'm just putting the finishing touches to my initial 1.0 release of the app (without recording), and will take a proper look at this towards the weekend, hoping to add it to the 1st update.

Can't wait to give this all a go, sounds much simpler than AudioQueue based recording!

Cheers.

Posted 2 years ago #
parsa28
Member
Hello Steve,

I need simultaneous recording and playback in an application that has a non-stop background music between various scenes and needs recording in one of the scenes. So I guess stopping denshion, running AVAudioRecorder and re-running denshion can't help me cause the bg music must be non-stop and "seamless". So I decided to use AudioQueue along with CDAudioManager.

What I currently do is to initialize, load buffers and play bg music in CDAudioManager. Then when the recording-enabled scene is loaded, I do (along with some other stuff):
```
AudioQueueNewInput(&format, listeningCallback, self, NULL, NULL, 0, &queue);

	UInt32 sessionCategory = kAudioSessionCategory_PlayAndRecord;
	AudioSessionSetProperty(kAudioSessionProperty_AudioCategory, sizeof(sessionCategory), &sessionCategory);
	UInt32 audioRouteOverride = kAudioSessionOverrideAudioRoute_Speaker;
	AudioSessionSetProperty (kAudioSessionProperty_OverrideAudioRoute,sizeof (audioRouteOverride),&audioRouteOverride);
```
As an adhoc solution It solves the problem to a very interesting point, but pauses the bg music for a fraction of a second or so and then continues both recording and playback. I've tried a lot of stuff to fix this but failed.

Do you have any Idea about how to fix this ? And if not and that I'm ought to write my own CDAudioManager to handle Play and Record category, where to start doing that ?

Thank you for your time dear Steve.

edit: I tried setting category for soundEngine (CDSoundEngine instance within CDAudioManager) to kAudioSessionCategory_PlayAndRecord on init but now all of my sounds are played at like half the original gain, but the recording works ! sounds weird.

edit #2: Well, it was just a matter of specifying the AudioRoute:
```
UInt32 audioRouteOverride = kAudioSessionOverrideAudioRoute_Speaker;
AudioSessionSetProperty (kAudioSessionProperty_OverrideAudioRoute,sizeof (audioRouteOverride),&audioRouteOverride);
```
So I guess thats all for having Recording along Denshion, at least for me :)
Posted 2 years ago #
Steve Oldmeadow
Administrator

@parsa28 - If you want simultaneous record + playback with background music I think you'll need to write your own engine based on RemoteIO (even lower level than Audio Queues). Check these links, particularly the first one. The author of the first link developed an app with simultaneous record and playback and had to go to RemoteIO to get satisfactory performance.

http://atastypixel.com/blog/2008/11/04/using-remoteio-audio-unit/
http://sites.google.com/site/iphonecoreaudiodevelopment/remoteio-playback

Posted 2 years ago #

RSS feed for this topic

Reply

You must log in to post.