Updated cocos3d – a 3D extension for cocos2d

A retina screenshot from the cocos3d Mashup Demo, insanely overloaded with mesh particles, text labels, animated robots, skinned hammers and running characters!

Update [2012-11-13]:

I’m pleased to announce that cocos3d 0.7.2 has been released.

This release adds heterogeneous 3D mesh particles, enhanced UIViewController integration, ray-casting enhancements, simplified parametric vertex array allocation and management, performance and memory usage improvements, enhanced logging, and detailed GL error tracing.

Features added in this 0.7.2 release include:

• Added mesh particles as a type of particle system.
  • Particles can now be constructed from any template mesh, (sphere’s, boxes, planes, POD models, etc).
  • A single emitter can support particles constructed from different template meshes.
  • Mesh particles can individually move, rotate and scale in three dimensions.
  • Particles can be added manually to an emitter, effectively creating a generic mesh batching system.
  • All mesh particle vertices are manipulated in main memory and submitted to the GL engine in a single GL call.
• Improved integration with UIView and UIViewController:
  • New CC3UIViewController class supports auto-rotation of the 3D/2D scene to the device orientation using standard UIKit auto-rotation.
  • Both iOS6 and pre-iOS6 rotation mechanisms are supported.
  • The CC3UIViewController loadView method now creates an appropriate CC3EAGLView, removing the need to create the view outside the controller in the AppDelegate.
• Added the ability to determine the individual mesh faces, and the exact mesh location, intersected by a ray (for example from a touch).
• Added the vertexContentTypes property to CC3MeshNode & CC3Mesh to simplify the creation of vertex arrays within a mesh. Parametric meshes (spheres, cones boxes, planes, etc.) can now be created with any type of vertex content.
• Added several new parametric mesh population methods, including parametric cones and invisible touch-boxes.
• Added support for bitmapped text labels as a 3D mesh.
• Added additional camera movement and framing methods.
• Added a build setting to provide the ability to trace GL errors to the exact GL call that triggered the error.
• Automatic handling of textures with pre-multiplied alpha channels.
• Memory and performance improvements including the use of minimal matrices, removal of many redundant GL calls, and the use of bitfields for most BOOLs.

Find out what’s new here.

cocos3d is a significant extension to cocos2d 1.x that adds a full 3D modelling space, including 3D mesh models, perspective projection cameras, materials, and lighting. With cocos3d, you can populate your world with 3D models exported from 3D editors such as Blender, 3ds Max or Cheetah3D, combined with models you create dynamically in your Objective-C code.

Integration with cocos2d 1.x is seamless. Rendering of all 3D model objects occurs within a special cocos2d layer, which fits seamlessly into the cocos2d node hierarchy, allowing 2D nodes such as controls, labels, and health bars to be drawn under, over, or beside 3D model objects. With this design, 2D objects, 3D objects, and sound can interact with each other to create a rich, synchronized audio-visual experience.

Like cocos2d, cocos3d is written entirely in Objective-C, and many of your favourite cocos2d paradigms, such as CCActions, are available for 3D objects. And like cocos2d, it is distributed for free under an MIT license.

cocos3d Features

Some of the key features of cocos3d are highlighted in the following list:

• Seamless integration with cocos2d 1.x. Rendering of all 3D model objects occurs within a special cocos2d layer, which fits seamlessly into the cocos2d node hierarchy, allowing 2D nodes such as controls, labels, and health bars to be drawn under, over, or beside 3D model objects. With this design, 2D objects, 3D objects, and sound can interact with each other to create a rich, synchronized audio-visual experience.
• Seamless integration with the iOS UIViewController framework.
• Pluggable loading framework for 3D models exported from familiar 3D editors such as Blender, 3ds Max or Cheetah3D, or through industry standard 3D object files such as Collada or PowerVR POD, or even from your own customized object file formats.
• 3D models can be selected and positioned by touch events and gestures, allowing intuitive user interaction with the objects in the 3D world.
• 3D models can include animation sequences, with full or fractional animation.
• 3D model objects can be arranged in sophisticated structural assemblies, allowing child objects to be moved and oriented relative to their parent structure.
• 3D models and assemblies can be easily duplicated. Each duplicated model can be independently controlled, animated, colored, or textured. But fear not, underlying mesh data is shared between models. You can quickly and easily create swarming hoards to populate your 3D world, without worrying about device memory limitations.
• 3D models, cameras, and lighting can be manipulated and animated using familiar cocos2d Actions, allowing you to quickly and easily control the dynamics of your 3D world, in a familiar, and easy-to-use programming paradigm.
• 3D objects can be covered with dynamic materials and textures to create rich, realistic imagery.
• Multi-texturing and bump-mapped textures are available, allowing you to create sophisticated surface effects.
• Vertex skinning, also often referred to as bone rigging, allowing soft-body meshes to be realistically deformed based on the movement of an underlying skeleton constructed of bones and joints.
• Automatic shadowing using shadow volumes.
• Collision detection between nodes.
• Ray-casting for nodes intersected by a ray, and the local location of intersection on a node or mesh, right down to the exact mesh intersection location and face.
• The 3D camera supports both perspective and orthographic projection options.
• Objects can dynamically track other objects as they move around the world. The 3D camera can dynamically point towards an object as it moves, and other objects can dynamically point towards the camera as it moves.
• Lighting effects include multiple lights, attenuation with distance, spotlights, and fog effects.
• Mesh data can be shared between 3D objects, thereby saving precious device memory.
• Mesh data can freely, and automatically, use OpenGL vertex buffer objects to improve performance and memory management.
• Culling of 3D objects outside of the camera frustum is automatic, based on pluggable, customizable object bounding volumes.
• Automatic ordering and grouping of 3D objects minimizes OpenGL state changes and improves rendering performance. Pluggable sorters allow easy customization of object sorting, ordering, and grouping for optimal application performance.
• Integrated particle systems:
  • 3D point particles provide efficient but sophisticated particle effects.
  • 3D mesh particles allow particles to be created from any 3D mesh template (eg- spheres, cones, boxes, POD models, etc).
• Automatic OpenGL state machine shadowing means that the OpenGL functions are invoked only when a state really has changed, thereby reducing OpenGL engine calls, and increasing OpenGL throughput.
• Sophisticated performance metrics API and tools collect real-time application drawing and updating performance statistics, for logging or real-time display.
• Sophisticated math library eliminates the need to use OpenGL ES function calls for matrix mathematics.
• Fully documented API written entirely in familiar Objective-C. No need to switch to C or C++ to work with 3D artifacts.
• Extensive logging framework to trace program execution, including all OpenGL ES function calls.
• Includes demo applications and Xcode templates to get you up and running quickly.

A Picture is Worth a Thousand Words

The following two screenshots are taken from the CC3DemoMashUp sample demo application that is included in the cocos3d distribution, and highlight some of the cocos3d functionality. The pictures show a scene taken at two different times.

The lighting is now dimmer, the robot arm has moved, and the 2D label is larger as it approaches the camera.

An animated robot arm, with a 2D label attached to the end, rotates amid teapots.

The scene includes:

• An animated rotating robot arm loaded from a PowerVR POD file. This POD file is taken from a sample demo that is bundled with the standard PVR SDK from Imagination Technologies. It demonstrates the ability to load and animate ‘out-of-the-box’ POD files. It also demonstrates how 3D nodes are assembled into a structural hierarchy (similar to cocos2d nodes), in that the arm has several parts that move relative to each other, while the complete robot assembly can be moved, rotated or scaled as a whole.
• A standard cocos2d text label that is attached to the end of the robot arm and rotates with the arm. This demonstrates the ability for 2D and 3D nodes to interact. This label could have been any cocos2D CCNode, even one you create yourself.
• Four Utah teapots built from the teapot.h header file distributed with the iOS SDK. The red, green and blue teapots are positioned at 100.0 on the X, Y and Z axes, respectively. The white teapot is positioned at the location of the light source for the scene, which is also animated as part of the POD file. The teapots demonstrate how cocos3d model objects can be created programmatically instead of loaded from 3D files. To conserve memory, the teapots all use the same underlying copy of mesh data, but each has a different material so they can be colored separately.
• A cocos2d logo positioned and scaled to act as a ‘floor’ or ‘ground’ to give the scene some spatial orientation. This logo is part of the 3D scene.
• Two green joystick buttons to control the panning and location of the 3D camera (the viewer’s perspective), and an arrow button to cycle the camera through three other scenes (by rotating the camera to the left). All three buttons are standard cocos2d nodes, and are added as children to the special CCLayer that is playing the 3D scene, demonstrating how 2D nodes can be overlayed on the 3D view. BTW…they joysticks are part of a small UI controls kit we created. You can find the kit here.

In the view on the left, the robot arm is rotating towards the back, with the 2D cocos2d label attached to the end. In the view on the right, the robot arm has rotated towards the front. The 2D label has come with it, and is scaled up automatically to create the proper perspective. This perspective scaling of 2D artifacts is optional. For something like a 2D health-bar hovering over a 3D character, you might want to keep it the same size, regardless of where the character is roaming.

Also in the second picture, the location of the light source (as marked by the white teapot), has moved down (think of a setting sun), and as a result, the lighting has dimmed, particularly on the side of objects facing away from the light.

In addition to the scene illustrated, the CC3DemoMashUp includes three other scenes, which can be viewed either by panning the camera, or by pushing the arrow button. These scenes are:

• A bouncing, multi-colored, semi-transparent beach-ball, which bounces courtesy of standard cocos2d CCActions. The beach-ball demonstrates a multi-material mesh model and transparency blending. It is particularly effective if you use the joysticks to move the camera behind the beach-ball and point the camera back towards the robot arm. You can see the robot arm through the beach-ball as it bounces by.
• Another two teapots, one textured and one with a color gradient applied to its vertices, rotating around each other.
• A rotating globe that demonstrates the ability to wrap a mesh with a UV mapped texture exported from a 3D editor.

The bouncing beach-ball and rotating globe were constructed in Blender, exported to a Collada 1.4 file format, and then converted to POD file format using the Collada2POD converter that comes with the PowerVR SDK. The ‘change view’ arrow button demonstrates the ability to assign a target location to cameras and lights. The button assigns a different target to the camera, and the camera pans (again using a cocos2d CCAction) to its new target.

More Info

You can download cocos3d here.

You can learn more about writing 3D iOS applications using cocos3d, by referring to:

• cocos3d homepage.
• cocos3d Programming Guide.
• cocos3d API.
• Demo applications in the Demos folder in your cocos3d distribution.
• cocos3d Development Roadmap.
• cocos3d Forum
• cocos3d Importing: Converting COLLADA to POD
• Check out a example video of a robot game created by cocos3d user andyman404 in less than a week!