© Copyright Khronos Group, Page 1 Bringing 3D Gaming to Cell Phones Being developed by the

© Copyright Khronos Group, Page 2 Topics Khronos and the OpenGL ES initiative -Neil Trevett, Secretary of the Khronos Group OpenGL ES API specification overview -David Blythe, OpenGL ES Specification Editor Demonstrations and industry support -Nokia, Hi, Motorola, PowerVR Call to action -Q&A Neil Trevett Khronos Secretary

© Copyright Khronos Group, Page 3 Large Opportunity for Graphics Software 3D capability is about to be everywhere 80’s 3D Hardware Acceleration in workstations 100Ks / annum 00’s 3D Hardware Acceleration in Handhelds 100Ms / annum Beach head application is 3D games on cell phones which will drive 3D acceleration hardware into handsets in ’s 3D Hardware Acceleration in everyday PCs 10Ms / annum

© Copyright Khronos Group, Page 4 Embedded Market has API Problem No suitable low-level graphics APIs exist - yet These APIs are the foundation for all advanced graphics applications It’s like having Desktop Windows without Direct3D or OpenGL Embedded Hardware Low-level Graphics APIs High Level Graphics Libraries Applications No suitable APIs exist at this level Games Engines JSR-184 Games Productivity Apps OpenGL is widely available across platforms – but is way too big ? Direct3D - not cross platform – so every platform other than Windows CE has a real problem ?

© Copyright Khronos Group, Page 5 Inverse Screen Size Law The need for advanced graphics Smaller screens need more advanced graphics processing per pixel -To make best use of restricted screen real-estate Advanced graphics techniques provided by state-of-the-art APIs -Anti-aliasing, multi-level compositing, rich media, 3D Interactive 3D Gaming Content Video Processing and synchronization Perspective windows maximize screen real-estate Anti-aliased text to make small text extremely legible Translucent Menus don’t obscure lower layers

© Copyright Khronos Group, Page 6 Cross Platform Solution Essential Many platforms and initiatives need these APIs WindML ® – expanding multimedia capabilities of VxWorks ® Looking to further expand 3D and rich media delivery capabilities Embedded Linux – need for low-footprint APIs No standardized 2D API other than X Windows – not low footprint Increasing use of OpenGL for 3D – but not low footprint Java Community – Java2D, Java Media Framework, Java Advanced Imaging Java3D is a high-level API – need low-footprint 3D API for embedded applications ARINC – standards for advanced cockpit displays Need standardized 2D & 3D APIs Severe certification demands drive need for minimum API size Web3D Consortium Extensible standards for 3D delivery over the internet, driving 3D in MPEG-4 BUT needs target small footprint 2D/3D API for higher-level standards Symbian OS for mobile consumer devices Needs minimal API size and increasing functionality Wireless Applications Platforms Need advanced graphics capability in small footprint for BREW and chipsets

© Copyright Khronos Group, Page 7 Working to define full- featured 3D API. Discussions with BroadcastCL group- to avoid divergence Strong support from ARINC and aerospace companies … At a meeting in April 2002, Symbian and the cell phone industry selected OpenGL ES as the graphics API for cell phones with accelerated 3D in 2004 Phones Web pads Cameras PDAs Hand-held Appliances Line-powered Appliances STBs Internet Terminals Consoles OpenGL ES Industry Momentum Safety Critical Avionics Automotive New Visual Processing APIs

© Copyright Khronos Group, Page 8 Khronos – an Open Standards Group Our Members are creating OpenGL-related APIs Working closely with the OpenGL ARB Promoting Members Contributing Members ARB Feedback and Ratification

© Copyright Khronos Group, Page 9 OpenGL ES A proposed embedded graphics API Create well-defined subset profiles of OpenGL -A series of increasingly compact profiles -Selecting functionality needed for key target markets ARB Feedback and Ratification Eliminate Redundancy Eliminate Pure Workstation Functionality Embedded Focus Workstation Focus

© Copyright Khronos Group, Page 10 OpenGL and OpenGL ES A close synergy OpenGL is already the most widely available cross-platform graphics API -Workstations to embedded appliances, PCs to Macs, Windows to Linux Strong future roadmap with OpenGL 2.0 -Enabling access to the new generation of programmable hardware Through Khronos – OpenGL ES be royalty free to the embedded industry -Agreed by SGI – MAJOR OPPORTUNITY Just two profiles - minimizing fragmentation -Common Profile -Minimum footprint full function 3D with texture-mapping -Good gaming platform -Implementable on cell phones -Safety Critical Profile -Absolute minimum 3D to ease safety certifications -Used in avionics and automotive displays Common Safety Critical The focus for the rest of today’s presentation

© Copyright Khronos Group, Page 11 API Overview David Blythe OpenGL ES Specification Editor

© Copyright Khronos Group, Page 12 Overview Where are we in OpenGL ES development? Handheld problem space Design goals What’s In – What’s Out Summary

© Copyright Khronos Group, Page 13 Where Are We? Handouts are a snapshot from January Presentation reflects specification as of March 4th Specification undergoing public review Specification still needs to be ratified July Khronos Participation agreement available at Siggraph August Ad hoc Working Group Meetings 4Q02 First draft specification 1Q03 “OpenGL ES 1.0” ratifiable by Khronos September Official Working Group Meetings

© Copyright Khronos Group, Page 14 Profile Design Goals Preserve OpenGL’s Goodness -Don’t try to fix OpenGL, it isn’t broken Proper subsets of OpenGL 1.3 -Match market space requirements Eliminate un-needed functionality -Redundancy -Expensive -Unused Leverage Workstation/PC experience -Extension mechanism, feature set Allow extensions to enable innovation -But should be the exception rather than the rule Defining conformance tests as well as the API -Minimum quality defined by conformance tests

© Copyright Khronos Group, Page 15 Handheld Problem Space Low power -Battery powered devices Low memory -50KB code budget Low resolution -120x120 display Rapidly evolving platforms -Cell phone, PDA, Set top, Wrist watch, ????

© Copyright Khronos Group, Page 16 Low Power Devices Low power consumption Modest processing power MHz integer processing -400Mhz PDA, 50MHz Cell phone, 10MHz Wrist Watch, 100MHz Set top Box Assume 32-bit integer Floating-point not available Battery power -Maximize battery life  minimize instruction/data traffic Where is the sweet spot ?

© Copyright Khronos Group, Page 17 Memory Footprint Platforms with limited storage -RAM MB, NAND FLASH MB Minimize library footprint -Reduce command footprint -Reduce number of ‘code paths’ Minimize data storage -Vertex data and attributes -Texture storage

© Copyright Khronos Group, Page 18 Floating-Point Arithmetic No HW floating-point in most of the target space -Today! Be both: -Integer friendly -Floating-point ready Key Issue Lots of Options -Multiple profiles -Fixed-point arithmetic -Floating-point emulation Lots of Tradeoffs -Robustness/Stability -Performance -Memory Footprint -API Complexity -Ease of Use

© Copyright Khronos Group, Page 19 Evolving Device Space Regular review of profiles -Target is yearly revision Profiles don’t become obsolete -Use with emerging devices Use OpenGL extension model -Enable HW vendors to ship extra features -Add to profile when demand is there Avoid creating fragmentation!

© Copyright Khronos Group, Page 20 Floating- and Fixed-point Critical Issue Want: -Efficient vertex data representation -Dynamic range for transformations -Accommodate platforms with and without floating-point Don’t want: -Redundant or seldom used code paths -API bloat -Confused application writers

© Copyright Khronos Group, Page 21 What’s In/What’s Out A Trip Down the OpenGL Pipeline Vertex & Geometry Processing Rasterization Texture Mapping Fragment Processing Buffer Operations Miscellaneous Operations Profile Structure Platform Embedding Utility Libraries

© Copyright Khronos Group, Page 22 Vertex and Geometry Processing InOut Vertex ArraysBegin/End, Edge Flags All PrimitivesTexGen Matrix StackUser Clip Planes Viewport, DepthRange Vertex LightingBack Materials Local Viewer, Secondary Color ShadeModel

© Copyright Khronos Group, Page 23 Rasterization InOut Multisampling (optional) Points and AA Points Lines & AA LinesLine Stipple PolygonsPolygon Stipple, Polygon Smooth

© Copyright Khronos Group, Page 24 Rasterization InOut Face CullingPolygon Mode PolygonOffset – fill modePolygonOffset – line, point mode Bitmap DrawPixels

© Copyright Khronos Group, Page 25 Texture Mapping InOut 2D Textures1D, 3D, Cube Maps, Proxy Wrap repeat, edge_clampclamp, border_clamp Texture Borders Compressed TextureGetCompressedTexImage TexSubImage, CopyTexImage MultitextureCombine Environment BGRA / packed formats, L, LAAll other image formats All Filters

© Copyright Khronos Group, Page 26 Fragment Processing InOut Fog Scissor Test Alpha Test Stencil Test (optional) Depth Test (required) Blending Logic Op Dither

© Copyright Khronos Group, Page 27 Framebuffer Ops InOut ClearAccum ReadPixelsCopyPixels Draw/ReadBuffer

© Copyright Khronos Group, Page 28 Miscellaneous InOut Flush/FinishEvaluators HintFeedback Selection Display Lists Get – static state (constants)Get – dynamic state Push/PopAttrib

© Copyright Khronos Group, Page 29 Profile Assumptions Lots of profiles doesn’t help -Creates support problems for developers -Tuned profiles not fully exploited Optional extensions are similar to multiple profiles Floating-point a medium-term trend -Eventually want both floating-point transforms and vertex attributes -Some platforms support it well now -Need to understand migration path now Want to cover today’s platforms well -Important for adoption Fixed-point is practical -Not ideal, but … -Successfully used in the past

© Copyright Khronos Group, Page 30 Profile Structure Two profiles for the entertainment and related spaces: -Mobile -PDA -Set top box -Consoles -Specialty devices, kiosks, terminals Start with profile that covers as much of this space as possible -Common profile Address less-capable platforms with subset -Common-light profile Vernacular -Integer = GLbyte, GLubyte, GLshort -Fixed-point = representation (GLfixed) -Floating-point = GLfloat (single-precision), GLdouble (double-precision)

© Copyright Khronos Group, Page 31 Common Profile Includes entire feature set just described -Integer + single-precision floating-point vertex attributes -Integer + single-precision commands as described in the spec -Double-precision floating-point commands where no pre-existing single-precision Adds fixed-point data type extension -Vertex attributes, command parameters -Pipeline implements floating-point dynamic range, even with fixed-point input Adds signed byte vertex data type extension -Vertex and texture attributes (normal, color already present) Adds extension to determine ‘implementation-specific’ read format -Minimize number of formats -Allow efficient read

© Copyright Khronos Group, Page 32 Common-Lite Profile Proper subset of common profile Currently, ‘one’ difference from common profile No floating-point (single- or double-precision) anywhere Command parameters, vertex data are fixed-point and integer Support contemporary and trailing-envelope platforms Support only one form of commands Nominally dynamic range in all computations (including transforms) -Subject to discussion No overflow checking or reporting -Subject to discussion Compile & run-time (dynamic libraries) with common profile

© Copyright Khronos Group, Page 33 Common Vs Common-Lite Common-Lite Common PDA Smart-phone Set Top Footprint Performance Power Consumption Game Device Kiosk Modest Handset

© Copyright Khronos Group, Page 34 Extensions Definition -Commands/features that change/extend the behavior of the OpenGL state machine -In the ‘gl’ name space Leverage current OpenGL extension mechanism -Extension string, function pointer query -ARB extensions -OpenGL ES extensions (OES) -Other OpenGL extensions -Vendor extensions Rules 1.Can’t re-add commands a profile removed (no begin/end) 2.Can’t create negative extensions (contractions) to remove features 3.Can add existing OpenGL extensions (e.G., Texture_cube_map) 4.Can add OpenGL ES working group (ESWG) exceptions to 1 -Candidates: texgen 5.Can add vendor-specific exceptions -Use to violate rule 1 

© Copyright Khronos Group, Page 35 Platform Embedding Interface OpenGL renderer to platform window system -Similar to GLX/AGL/WGL Issues: Platform specific or not -Is there significant value in a cross-platform API Feature set -Minimum platform capabilities -Video, p-buffers, render-to-texture

© Copyright Khronos Group, Page 36 Packaging Retain ‘GL’ namespace Single C header file -#defines for versions, profile variants -GL_ES_PROFILE_XX -GL_ES_PROFILE_XX_1_0 -XX = CM, CL, SC Runtime queries -GetString(GL_VERSION) OpenGL ES -XX 1.0 -XX = CM, CL, SC, … -GetString(GL_EXTENSIONS) “foo_OES bar_OES …” Proprosal for link/runtime library packaging -libGLESXX.{a,so,lib} -Versioning uses platform native versioning mechanism

© Copyright Khronos Group, Page 37 Utility Libraries No GLU subset -Project, Unproject, perspective useful, not hard to implement -Quadrics – questionable value (demo programs) -Tessellator, NURBS too large GUI libraries -Run away!

© Copyright Khronos Group, Page 38 Summary Embedded market space is very diverse -Focusing on handheld and safety-critical markets Creating OpenGL profile that enable handheld devices -Small footprint -Software renderers -Integer/fixed-point Evolving standard -Yearly revisions Draft spec undergoing public review -Now is the time to comment

© Copyright Khronos Group, Page 39 Demonstrations

© Copyright Khronos Group, Page 40 Demonstrators Proto OpenGL ES Implementations Kari Pulli – Nokia Mark Callow – Hi Corporation Mike Olivarez – Motorola Mitsubishi MBX and PowerVR Technology overview – Khronos Booth

© Copyright Khronos Group, Page 41 OpenGL ES Hardware Support The technology is here now! -New hardware generation for mobile platforms. MBX and PowerVR Technology overview -Tile Based Rendering and Deferred Rendering More information at: Advanced Graphic Hardware Embedded Software Developers OpenGL ES Advanced Consumer Content

© Copyright Khronos Group, Page 42 OpenGL 2.0 “Pure” OpenGL 2.0 subset OpenGL ES 2.0 OpenGL Standards Universe How things COULD develop OpenGL 1.3 Aiming for Ratification by Siggraph 2003 OpenGL 2.0 includes full 1.4 functionality – applications run unmodified OpenGL 1.4 GL2 extensions Applications can incrementally can use shading language extensions OpenGL ES 1.0 “Pure OpenGL 2.0” subset uses programmability to form a simple, compact, efficient and flexible API Full shader programmability for embedded devices

© Copyright Khronos Group, Page 43 Call to Action Your chance to influence OpenGL ES! Join Khronos – as a Promoter or Contributor -Have a voice in OpenGL’s development -Reasonable annual fee plus participation agreement -Talk to Elizabeth Riegel, the Khronos Group Manager here at the session, on our booth, or via at Review the draft specification – we are looking for ISV volunteers -Simple review agreement required to participate - Information at -Or the Secretary at

© Copyright Khronos Group, Page 44 Any Questions? OpenGL is a registered trademarks of Silicon Graphics, Inc., and OpenML is a trademark of Silicon Graphics, Inc., used with permission by the Khronos Group. All other product names, trademarks, and/or company names are used solely for identification and belong to their respective owners.