Dear All,
I'd like to include a sound wave in my game. How can I create it?
Here is an example of what I'd like to achieve:
Thank you very much!
How to create a soundwave?
(10 posts) (3 voices)-
You should check out how to do custom OpenGL drawing for that.
Note that there is a difference in OpenGL ES 1.1 and 2.0 regarding the GL functions used to draw as well as other stuff. If you are using Cocos2d 1.x they you should learn about using OpenGL ES 1.1, and Cocos2d 2.0 use GLES 2.0.
For OpenGL ES 1.1, I learned how to create something similar to this using Apple's AurioTouch sample code that draws the time domain and frequency domain data of a sound wave as well as a sonogram. Complicated app but lots can be learned from it if you take the time. You can download the AurioTouch app from Apple's sample code projects at developer.apple.com
Don't know of an OpenGL ES 2.0 sample project though that does the same.
After you figure out how to do custom drawing, you can create a CCNode and override the draw method to create your sound wave in Cocos2d.
Posted 9 months ago # -
mm24
MemberCool, that's exactly what I was looking for. I am using Cocos2d v2.0. I will spend some days trying to learn this as I guess is a "must know" to craft the game I have in mind.
The code looks like a lot to learn and know and I hope I will manage. And I guess once I have done so I'll be able also to understand how to detect collision with other objects present in the scene (I guess that the bbox of the effect CCNode would not apply and I'll have to do some interpolation between the bbox of the entity sprite and the pixels where the modified node is drawing the effect).
Thank you very much :)
PS: Also, do you think that using shaders will drop my game performance when compared to having a similar animation via a series of sprite frames?
Posted 9 months ago # -
Be careful to distinguish between GLES 1.1 and GLES 2.0 setup and functions. I think the AurioTouch and AurioTouch2 sample code from Apple both use GLES 1.1, and that won't work if you are using Coco2d v2.0.
Be careful messing with custom drawing in Cocos2d so you don't accidentally break functionality in Cocos2d that you need.
Regarding performance, the best thing to do is try it see what happens. In general, passing work from the CPU to the GPU will improve performance, but there are lots of factors that can skew that so testing is the only way to know for certain.
In Coco2d 2.0 which uses OpenGL ES 2.0, here is an example of a custom draw method.
Note the use of the shader program that is built into Cocos2d to make things easier and not screw up Cocos2d's behavior after custom drawing completes.
The glDrawMode is my custom enum.
-(void) draw { if (shouldDrawDynicVerts == YES) { ccGLUseProgram( shaderProgram->program_ ); [shaderProgram setUniformForModelViewProjectionMatrix]; ccGLEnableVertexAttribs( kCCVertexAttribFlag_Position); glVertexAttribPointer(kCCVertexAttrib_Position, 3, GL_FLOAT, GL_FALSE, 0, dynamicVerts); glEnableVertexAttribArray(kCCVertexAttribFlag_Position); glVertexAttribPointer(kCCVertexAttrib_Color, 4, GL_UNSIGNED_BYTE, GL_TRUE, 0, dynamicVertColors); glEnableVertexAttribArray(kCCVertexAttribFlag_Color); if (glDrawMode == kDrawTriangleStrip) { glDrawArrays(GL_TRIANGLE_STRIP, 0, dynamicVertCount); }else if (glDrawMode == kDrawLines){ glLineWidth(1); glDrawArrays(GL_LINES, 0, dynamicVertCount); }else if (glDrawMode == kDrawPoints){ glDrawArrays(GL_POINTS, 0, dynamicVertCount); }else if (glDrawMode == kDrawTriangleFan){ glDrawArrays(GL_TRIANGLE_FAN, 0, dynamicVertCount); } } }Here is a sample draw method in Cocos2d 1.x that uses OpenGL ES 1.1.
Note the need to return to the default GL states so that Cocos2d will function normally after doing your custom GL drawing.
Also note the difference between the functions used in 2.0.
There is also extra stuff I am doing here in this draw method, like using a frame buffer object in some of the drawing... thought I would post it since you might later find it useful.-(void) draw { // Dynamic verticies that are not in a FBO. if (levelStaticObjects.drawDynamicLevelBuildingVerts) { // Default GL states: GL_TEXTURE_2D, GL_VERTEX_ARRAY, GL_COLOR_ARRAY, GL_TEXTURE_COORD_ARRAY // Needed states: GL_VERTEX_ARRAY, // Unneeded states: GL_TEXTURE_2D, GL_TEXTURE_COORD_ARRAY, GL_COLOR_ARRAY glDisable(GL_TEXTURE_2D); glDisableClientState(GL_TEXTURE_COORD_ARRAY); glBindBuffer(GL_ARRAY_BUFFER, 0); //glFrontFace(GL_CW); GLuint stride = sizeof(tVertPlusColor4B); // Draw layer front const GLvoid* colorPointer = (GLvoid*) ((HeyaldaPoint*)levelStaticObjects.dynamicLevelBuildingVerts[2] + 1); glVertexPointer(3, GL_FLOAT, stride, levelStaticObjects.dynamicLevelBuildingVerts[2]); glColorPointer(4, GL_UNSIGNED_BYTE, stride, colorPointer); glDrawArrays(GL_TRIANGLE_STRIP, 0, levelStaticObjects.dlbvIndex[2]); glEnableClientState(GL_TEXTURE_COORD_ARRAY); glEnable(GL_TEXTURE_2D); } if (levelStaticObjects.islevelDrawArraysReady == NO) { return; } // Default GL states: GL_TEXTURE_2D, GL_VERTEX_ARRAY, GL_COLOR_ARRAY, GL_TEXTURE_COORD_ARRAY // Needed states: GL_VERTEX_ARRAY, // Unneeded states: GL_TEXTURE_2D, GL_TEXTURE_COORD_ARRAY, GL_COLOR_ARRAY glDisable(GL_TEXTURE_2D); glDisableClientState(GL_TEXTURE_COORD_ARRAY); GLuint stride = sizeof(tVertPlusColor4B); const GLvoid* colorOffset = (GLvoid*) sizeof(HeyaldaPoint); // Draw the verticies that are in a frame buffer object glBindBuffer(GL_ARRAY_BUFFER, levelStaticObjects.groundLayerHandle[2].drawHandle); glVertexPointer(3, GL_FLOAT, stride, 0); glColorPointer(4, GL_UNSIGNED_BYTE, stride, colorOffset); glDrawArrays(GL_TRIANGLE_STRIP, 0, levelStaticObjects.groundLayerHandle[2].totalFBOSizeInArrayElements); glBindBuffer(GL_ARRAY_BUFFER, 0); glEnableClientState(GL_TEXTURE_COORD_ARRAY); glEnable(GL_TEXTURE_2D); }Posted 9 months ago # -
mm24
MemberHi, thanks. I am kind of blocked. I got scared by OpenGL ES 2.0 vs ES 1.0 and the fact that some people suggested me to start with ES 1.0 and roll back to Cocos2d v1.0.
I have still 2-3 months of development time for my game and I wanted to make the best possible effort. Although, I don't understand why something written in a draw method might mess up with the rest of the game performance.
I just made a list on my wall of features that I need to add to my game, separating items between graphical and not. Part of my brain tells me "learn ES 1.0 first" but then I say "I have just moved to Cocos2d v2.0 and it doesn't seem possible", then the brave part says "you can do it, learn ES 2.0 with raywenderlich.com tutorials.. but then I think "omg, that's too much and too slow, should I develop all the other game features first and then do learn this"?
What would bring me to learn those things first is that I am not sure how the collision detection with bullets will change having a waveform bullets, and I am afraid to have to then change lots of code.
I feel so stupid having wasted 3 days doing nothing apart of asking questions and not yet trying anything. What are your thoughts on this and on the migration between Cocos2d v2.0 and v.1.0?
Thanks and sorry to bug you all with those lame considerations.
Posted 9 months ago # -
You might want to experiment with creating maybe 100 small sprites that you move to simulate the waveform. That might be easier to accomplish but would probably be less efficient.
Not sure what you mean by waveform bullets. Are you hoping to somehow have physics interact with the waveform?
Regarding something written in the draw method messing up Cocos2d
In Cocos2d 1.x with OpenGLES 1.1, you need to make sure that you restore the default draw state so that when Cocos2d continues to draw sprites for you it will have OpenGL in the correct state.In OpenGLES 2.0 you the main thing seems to be which shader program will be used. I just use the ones that Cocos2d has cached to make life simple. Looks like Cocos2d 2.0 uses the CC_NODE_DRAW_SETUP() macro before drawing things like layers and sprites to ensure the correct draw program is ready.
self.shaderProgram = [[CCShaderCache sharedShaderCache] programForKey:kCCShader_PositionColor];If you try to get more advanced, there probably is other OpenGLES functionality that could cause strange results in the way Cocos2d draws if enabled or disabled. Just need to test to make sure you don't break something.
Choosing Coco2d 1.x and 2.0
If you are already using Coco2d 2.0, you probably should stick with it unless you can identify some significant reason to change to Cocos2d 1.x. Why do people tell you to start with OpenGLES 1.1? I do not think that OpenGLES 2.0 is any more difficult than OpenGLES 1.1; it is just different. In Cocos2d 2.0 learning OpenGLES 2.0 you do not even need to mess with shader programs if you don't want to. You can just use the ones built into Cocos2d 2.0. For a sound wave, you likely will just need the shader Cocos2d caches by the key kCCShader_PositionColor.Some more detail about the OpenGLES 2.0 code I posted above
The glDrawMode is my own custom enum to decide which draw method to use.
The dynamicVerts is a pointer to an C array of a 3 float structs, but you could replace it with CGPoint and change the glVertexAttribPointer parameter from 3 to 2 to set it up for two dimensions instead of three. This array defies your geometry that you will draw.
The dynamicVertColors is a pointer to a C array of ccColor4Byte structs. This array aligns to vertices in glVertexAttribPointer to color what is drawn.
Posted 9 months ago # -
mm24
MemberHi,
thank you for your reply and explanation. I tried to implement it but I am having some troubles. I added the following draw code in the draw() method of a new CCNode subclass:
-(void) draw { CCLOG(@"Calling draw"); GLfloat squareVertices[12] = { -0.5f, -0.33f, 0.0, 0.5f, -0.33f, 0.0, -0.5f, 0.33f, 0.0, 0.5f, 0.33f, 0.0 }; ccColor4B dynamicVertColors[12] = { ccc4(255, 255, 244, 255), ccc4(255, 255, 244, 255), ccc4(255, 255, 244, 255), ccc4(255, 255, 244, 255), ccc4(255, 255, 244, 255), ccc4(255, 255, 244, 255), ccc4(255, 255, 244, 255), ccc4(255, 255, 244, 255), ccc4(255, 255, 244, 255), ccc4(255, 255, 244, 255) }; enum glDrawMode drawMode = kDrawLines; // ccGLUseProgram( shaderProgram_->program_ ); // [shaderProgram_ setUniformForModelViewProjectionMatrix]; ccGLEnableVertexAttribs( kCCVertexAttribFlag_Position); glVertexAttribPointer(kCCVertexAttrib_Position,3, GL_FLOAT, GL_FALSE, 0, squareVertices ); glEnableVertexAttribArray(kCCVertexAttribFlag_Position); glVertexAttribPointer(kCCVertexAttrib_Color, 4, GL_UNSIGNED_BYTE, GL_TRUE, 0, dynamicVertColors); glEnableVertexAttribArray(kCCVertexAttribFlag_Color); if (drawMode == kDrawTriangleStrip) { glDrawArrays(GL_TRIANGLE_STRIP, 0, 4); }else if (drawMode == kDrawLines){ glLineWidth(1); glDrawArrays(GL_LINES, 0, 4); }else if (drawMode == kDrawPoints){ glDrawArrays(GL_POINTS, 0, 4); }else if (drawMode == kDrawTriangleFan){ glDrawArrays(GL_TRIANGLE_FAN, 0, 4); } }Unfortunately I couldn't use:
ccGLUseProgram( shaderProgram->program_ );so I substituted it with this: shaderProgram_
But when running it was giving a BAD ACCESS memory error.I also couldn't get the code to work with a 2D array of CGPoint. Finally I wasn't sure how to deal with GLsizei and hence I put 4 as default as I have an array of 12 floats forming (in theory) 4 square vertices.
Unfortunately the result is a black screen.
Is there anything that I am doing clearly wrong?
Thanks!
Posted 9 months ago # -
I was using Coco2d 2.0 rc2 or rc0a and needed to add the shaderProgram instance variable because it did not yet exist in the CCNode code, but in Cocos2d 2.0 it was added to the CCNode as an instance variable. You still need set it though or you will get a bad access crash. You can set it with one of the shader programs in sharedShaderCache or you can create your own shader programs.
Assuming that you have not manipulated the camera for your node, try increasing the size of the vertices that you define. Right now you are trying to draw four lines that will look like a backwards z, but position of the verts are so close together you likely will only get a single pixel dot that is partly off the screen.
The coordinates that you are defining are node relative, with the bottom left corner of the node at 0,0. So if you set the anchorpoint of the node to be 0,0 and position the node at 0,0 then you can have the whole screen as your canvas to draw. For example, on an iphone in landscape your coordinate plane would be 480x320.
I am going to try to create a blog post at http://heyalda.com/blog with sample code for drawing in OpenGL 2.0 today. Will publish by early next week or maybe even today.
Posted 9 months ago # -
As promised, I created a tutorial on how to implement custom OpenGL ES 2.0 drawing in Cocos2d. The tutorial includes a sample project including a helper class that can be used to batch drawing.
The sample project demonstrates how to create a sound wave using GL_LINE_STRIP, how to draw with GL_TRIANGLE_STRIP, and a bunch of other stuff.
@mm24, to achieve a sound wave exactly like the image you posted, you could try using lots of vertices or maybe a custom shader program would do the trick.
Let me know what you think.
http://heyalda.com/drawing-with-opengl-es-2-0-in-cocos2d-2-0/
Jim
Video of the sound wave simulation:
http://www.youtube.com/watch?feature=player_embedded&v=YGt4ZfX1C6gPosted 9 months ago #
Reply
You must log in to post.