Advanced Graphics Algorithms Ying Zhu Georgia State University 3D Graphics on Mobile Devices

Applications Games Map visualization and navigation User interface Animated message Screen savers

The challenges The challenges of developing 3D graphics content for mobile devices Battery power Heat management (no room for fans) Small, low resolution displays CPU Memory Slow system bus Download size limits Input devices Sound

The challenges The challenges of developing 3D graphics content for mobile devices (continued) No floating point hardware No integer divide hardware Multi-tasking (phone calls, texting, etc.) Device fragmentation (different operating systems, different graphics chips, different architectures, different input devices)

3D graphics APIs for mobile devices Image courtesy of “Mobile Graphics” course at ACM SIGGRAPH 2007

OpenGL ES “OpenGL® ES is a low-level, lightweight API for advanced embedded graphics using well-defined subset profiles of OpenGL.” “It provides a low-level applications programming interface (API) between software applications and hardware or software graphics engines.”

OpenGL ES OpenGL ES is subset of OpenGL specification Eliminated functions that are not suitable for mobile devices Lightweight and low footprint

OpenGL ES and OpenGL Not in OpenGL ES Kept in OpenGL ES 1.0 and 1.1 glBegin() and glEnd() Vertex array Polygons and quads Points, lines, triangles GLU functions Most of the lighting related functions Display lists, accumulation buffer 3D texture mapping, cube mapping 2D texture mapping, multitexturing, texture filter glPushMatrix() and glPopMatrix() Transformations DrawPixels, PolygonMode, polygon stipple, anti-aliased polygon rendering, etc. Vertex buffer objects, etc.

OpenGL ES OpenGL ES pipelines http://people.csail.mit.edu/kapu/siggraph_2007/mob3D_SG07.pdf

OpenGL ES 2.0 Released in 2007 Replaced the fixed function pipeline with programmable shaders Transformation and lighting functions are replaced by shaders OpenGL ES 2.0 is not backward compatible with OpenGL ES 1.1

OpenGL ES 2.0

OpenGL ES 2.0

Software implementation Reference implementation http://www.khronos.org/opengles/documentation/gles-1.0c.tgz

Hardware implementation Nvidia Sony PSP AMD ARM Toshiba Mitsubishi and more …

OpenGL ES usage OpenGL ES 1.0 OpenGL ES 1.1 OpenGL ES 2.0 Official 3D graphics API for Symbian, Android, and Playstation 3 OpenGL ES 1.1 3D library for iPhone SDK OpenGL ES 2.0 3D library for Pandora console

Mobile 3D Graphics API (M3G) M3G is an optional API for Java ME The commonly used version is JSR-184 The latest version is JSR-297 Can be implemented on top of OpenGL ES M3G for Java ME is defined in package javax.microedition.m3g Can be quite slow compared with OpenGL ES or Mascot Capsule applications

M3G M3G is a scene graph based API E.g. objects and transformations are organized in a tree hierarchical structure A high level API (e.g. supports hierarchical transformation and animation)

M3G Defines a scene graph structure World (the root) Node Object3D Camera Group Light Mesh SinnedMesh Sprite3D MorphingMesh

M3G Defines a file format for managing and deploying 3D content Provides a Loader class to download and deserialize graph nodes and node components, as well as entire scene graphs 

M3G Graphics3D A singleton 3D graphics context that can be bound to a rendering target. All rendering is done through the render() methods in this class. The most important class

M3G Other features supported by M3G VertexArray Transform Material Texture2D RayIntersection Fog AnimationController AnimationTrack KeyframeSequence

Development tools NetBeans Mobility EclipseME And many other tools … http://mobility.netbeans.org/ EclipseME http://sourceforge.net/project/showfiles.php?group_id=86829 And many other tools …

Other mobile graphics APIs Mascot Capsule A real-time 3D rendering engine for mobile devices Emphasis on performance (faster than M3G) Used mostly in Japan http://www.mascotcapsule.com/en/index.php Windows Mobile DirectX and Direct3D http://msdn.microsoft.com/en-us/library/ms172504.aspx

Resources ACM SIGGRAPH 2007 course “The Mobile 3D Ecosystem” http://people.csail.mit.edu/kapu/siggraph_2007/ IEEE Computer Graphics and Applications, Vol. 28, No. 4, 2008 special issue on Mobile Graphics “Mobile Computer Graphics” course at Lund University, Sweden http://www.cs.lth.se/EDA075/

Resources OpenGL ES Mobile 3D Graphics API (M3G) http://www.khronos.org/opengles/ Mobile 3D Graphics API (M3G) JSR 184 (http://www.jcp.org/en/jsr/detail?id=184) JSR 297 (http://www.jcp.org/en/jsr/detail?id=297) Kari Pulli, Tomi Aarnio, Ville Miettinen, Kimmo Roimela, Jani Vaarala: Mobile 3D Graphics with OpenGL ES and M3G, Morgan Kaufmann, 2007