1. © Copyright Khronos Group, 2002 - Page 1 BOF Meeting SIGGRAPH 2002 www.khronos.org neil.trevett@3dlabs.com Khronos Secretary

2. © Copyright Khronos Group, 2002 - Page 2 Today’s Agenda Khronos - Technical and Organizational Overview What is Khronos? What are our goals? -Neil Trevett, Senior Vice President, 3Dlabs - Khronos Secretary OpenML™ 1.0 Technical Overview -Jean-Luc Dery, Technical Leader, Advanced Systems, Discreet OpenGL ® ES - Embedded API initiative -Neil Trevett, Senior Vice President, 3Dlabs - Khronos Secretary Khronos IP Policy – an Innovative Approach -Shawn Underwood, Director of Marketing, SGI How do I get involved in driving Khronos Standards? -Neil Trevett, Senior Vice President, 3Dlabs - Khronos Secretary Questions and Answers – we want your feedback!

3. © Copyright Khronos Group, 2002 - Page 3 Driving OPEN Standards for CREATING and DEPLOYING DYNAMIC Media

4. © Copyright Khronos Group, 2002 - Page 4 Khronos’ Evolving Organization Khronos 1.0 Original Promoters cooperated under Khronos Participation agreement 1.0 to produce OpenML 1.0 specification Khronos 2.0 New Khronos Participation Agreement 2.0 Recruiting member companies now Enables multiple working groups Media API for Dynamic Media Authoring. Specification available now Deploying and evolving OpenML Open GL ES Small footprint OpenGL profiles for Embedded Systems

5. © Copyright Khronos Group, 2002 - Page 5 Khronos Goal #1: Simplify Dynamic Media AUTHORING Publicly Released at Siggraph 2001 Publicly Released at Siggraph 2001 The result…

6. © Copyright Khronos Group, 2002 - Page 6 Dynamic Media Programming Synergistic integration of OpenML and OpenGL What OpenGL is to GRAPHICS - 2D, 3D and image processing… …OpenML is to MEDIA - video and audio processing OpenML and OpenGL together create a unified programming environment -With seamless integration of graphics and media Tightly integrated graphics, video and audio = DYNAMIC MEDIA Workstation 3D graphics Professional video and audio authoring

7. © Copyright Khronos Group, 2002 - Page 7 ? OpenML Digital Media Programming Environment Applications Window System System OpenGL ® MLdcML 2D Graphics3D GraphicsDisplays Video Audio Synchronization OpenGL Extensions

8. © Copyright Khronos Group, 2002 - Page 8 Technical Strategy for OpenML 1.0 Move PROVEN technologies into Open Standards UST/MSC Synchronization #4: synchronization of diverse media streams (graphics, audio, video) SGI’s dmSDK Digital Media Handling #1: Configure and control the Input/Output and processing of video and digital audio ML OpenGL Extensions Integrating graphics and video #3: Enable OpenGL hardware to add more value to the media authoring workflow SGI’s Xdc extension to X11 Display Control #2: Configure and control display devices MLdc Problem Solution

9. © Copyright Khronos Group, 2002 - Page 9 Khronos Goal #2: Embedded Devices and Appliances Create Standards for Dynamic Media on The result… OpenGL ® ES

10. © Copyright Khronos Group, 2002 - Page 10 Well Defined OpenGL Profiles Addressing broad application requirements Khronos activities are endorsed and supported by the OpenGL ARB 2D 3D OpenGL ES ARB Feedback and Ratification Workstation FocusMedia and Embedded Focus Small footprint OpenGL profiles

11. © Copyright Khronos Group, 2002 - Page 11 OpenML 1.0 Technical Overview Jean-Luc Dery Technical Leader, Advanced Systems Discreet, a division of Autodesk

12. © Copyright Khronos Group, 2002 - Page 12 Overview What is OpenML? Open and cross-platform standard Programming environment used for capturing, processing, synchronizing and playing digitial media content (video and audio) Composed of four components: -Graphics video backend control (MLdc) -Video/Audio IO (ML) -OpenGL extensions -Synchronization (UST/MSC) Application MLdc Video channels Graphics devices Digital media devices OpenGL + Extensions ML

13. © Copyright Khronos Group, 2002 - Page 13 OpenML’s “MLdc” API Display control interface Enables an application to control the video back-end of the graphics devices that produce video output streams Enables to set up channels in the display screen(s) -Channels that are driven from sections of physical frame buffers -Channels depends on the display device Frame Buffers Progressive Scan Computer Monitor Progressive Scan Computer Monitor Progressive Scan HDTV Monitor Progressive Scan HDTV Monitor Interlaced Video Monitor Interlaced Video Monitor Channels Screens

14. © Copyright Khronos Group, 2002 - Page 14 OpenML’s “MLdc” API (ctn’d) MLdc provides control for: Screens and channels Quering monitor information Video output channel position Video format query and settings Vertical retrace rate Scan mode (interlaced vs. progressive) Gamma ramp control Video synchonization sources Genlock state notification

15. © Copyright Khronos Group, 2002 - Page 15 OpenML’s “ML” API Media device control interface Low level API that allows to configure and control video/audio device input, output and processing features Allows to setup processing pipelines between application and I/O devices -Asynchronous communication between application and media devices -Device control and capability queries -Synchronization mechanism -Processing capability for digital media streams -Optimized buffering mechanism for smooth and efficient delivery of media streams OpenInventorQuickTime OpenGLML Low level (ioctl interface) 3DMedia

16. © Copyright Khronos Group, 2002 - Page 16 OpenML’s “ML” API (ctn’d) Application and memory centric model Application memory Video input/output device Video input/output device Graphics 1. send 2. receive 3. drawPixels

17. © Copyright Khronos Group, 2002 - Page 17 OpenML’s OpenGL Extensions Video imaging and 3D graphics integration OpenML is built around OpenGL OpenML and OpenGL: A tight and synergistic integration Extends OGL with video primitives and DCC rendering features Defines a set of OpenGL extensions that OpenML compliant hardware must implement

18. © Copyright Khronos Group, 2002 - Page 18 OpenML’s OpenGL Extensions (ctn’d) Current extensions Pbuffers are the fundamental method of integrating graphics and video -Enables buffering and off-screen processing of video data -The primary data link between OpenML and OpenGL OpenGL imaging extensions are used to process video streams -Filtering, color correction, blending, image enhancement, color space conversion Synchronization control -Primitives to control UST/MSC synchronization from OpenGL -Functions added: WaitForMSC, SwapBuffersMSC, GetSyncValues Interlace support -Based on two previous extensions: SGIX_interlace, INGR_interlace_read Enables reading and writing of the frame buffer while skipping every other line -Enables processing (e.g., convolution) on interlaced video streams

19. © Copyright Khronos Group, 2002 - Page 19 OpenML’s OpenGL Extensions (ctn’d) Current extensions Texture color mask management to enable advanced compositing -Enables independent loading of RGB images and Alpha masks in texture memory -Recently promoted from an SGI extension to an ARB extension -Works for textures like the OpenGL color mask works for the color buffer Chroma resampling to support pixel formats such as YCrCb -Upsampling and downsampling when converting between YCrCb and RGB -Sampling options are: Replicate, Zero fill, Average

20. © Copyright Khronos Group, 2002 - Page 20 OpenML Synchronization Media device synchronization interface Critical part of authoring environment Allows to synchronize video, audio and graphics streams making sure that all media streams start at the same time and stay in sync Non-trivial problem to solve -Devices from various manufacturers -Device latencies are unpredictable -OS scheduling latencies Audio Device A Video Device A Audio Device B Gfx Device B Gfx Device A Application Do all the streams come out in sync?

21. © Copyright Khronos Group, 2002 - Page 21 OpenML Synchronization (ctn’d) The UST/MSC/SBC Solution UST - Unadjusted System Time – time reference for s/w and devices -64-bit monotonic counter with nanosecond resolution, microsecond system-wide accuracy MSC - Media Stream Counter - incremented for each channel sample -Video - per frame, Graphics - per retrace, audio - per audio sample, e.g. 44.1KHz SBC – Swap Buffer Count - incremented on graphics buffer swap -Not necessarily in sync with retrace UST Graphics devices Digital audio/ video devices SBC OpenGL ML MSC Output graphic frames/ samples Video/Audio IO samples

22. © Copyright Khronos Group, 2002 - Page 22 OpenML Synchronization (ctn’d) Robust synchronization – even on non real-time OS Every media sample is time-stamped -Timestamp can be accessed through OpenGL and ML APIs for temporal analysis and control Application can synchronize output events to UST -Enabling precise timing and synchronization in non-real-time operating systems Applications can detect dropped samples -And take corrective action

23. © Copyright Khronos Group, 2002 - Page 23 Real-world Examples Currently shipping Discreet applications on PC, Mac and SGI -Special effects and compositing -Off-line and on-line editing -Broadcast graphics -3D modeling and animation Scenarios examples -Video preview -Media playback -Video capture -Real-time video effects Expertise learned from these products was used to define the OpenML standard

24. © Copyright Khronos Group, 2002 - Page 24 Benefits of OpenML To Developers… OpenML will allow multimedia application developers to… -Spend more time and energy on application development and less time on mundane details such as device control, synchronization and buffering issues -Avoid dealing with different APIs for video, graphics and audio for each operating environment -Develop and deploy efficient cross-platform applications that interoperate a wide selection of systems and peripherals OpenML will allow hardware vendors to… -Gain access to new market opportunities by delivering on the promise of interoperability -Leverage the efforts of a larger base of application developers -Add significantly more value to the rich media creation workflow

25. © Copyright Khronos Group, 2002 - Page 25 Future Developments - OpenML 1.1 Meeting the ongoing needs of media authoring Tight integration into the evolving OpenGL 1.1 framework Closer integration of ML and OpenGL video streaming Higher quality rendering OpenGL resource management Conformance tests and performance benchmarks POSSIBILITIES include: -Extended color range >8 bit per component -Improved compositing capabilities – separate alpha and matte channels -Improved texture and memory management – including render to texture, interlaced textures -Improved efficiency – including asynchronous operations -Improved live video handling - streaming of video to/from Pbuffers We need your input now – particularly application developers!

26. © Copyright Khronos Group, 2002 - Page 26 Any Questions on OpenML?

27. © Copyright Khronos Group, 2002 - Page 27 Want to use OpenML 1.0? Application Developers No license needed to use OpenML No license needed to use OpenML Specification, free of charge, at www.khronos.org Specification, free of charge, at www.khronos.org OpenML Library Implementers Royalty-free Adopters Agreement Royalty-free Adopters Agreement Spec and agreement at www.khronos.org Spec and agreement at www.khronos.org

28. © Copyright Khronos Group, 2002 - Page 28 Encouraging OpenML Deployment OpenML Implementation Working Group Create sample implementation -Starting from SGI’s donated dmSDK source Define DDI level interfaces -To enable new devices to be added easily Cooperative, possibly open source, development -Source trees managed by Khronos Define and implement conformance tests -Maintain specification integrity and ensure portability

29. © Copyright Khronos Group, 2002 - Page 29 OpenGL ES Overview Neil Trevett Vice President, 3Dlabs

30. © Copyright Khronos Group, 2002 - Page 30 Khronos Goal #2: Embedded Devices and Appliances Create Standards for Dynamic Media on The result…

31. © Copyright Khronos Group, 2002 - Page 31 Inverse Screen Size Law The need for advanced graphics Smaller screens need more advanced graphics processing per pixel -To make best use of restricted screen real-estate Advanced graphics techniques provided by state-of-the-art APIs -Anti-aliasing, multi-level compositing, rich media, 3D Interactive 3D Gaming Content Video Processing and synchronization Perspective windows maximize screen real-estate Anti-aliased text to make small text extremely legible Translucent Menus don’t obscure lower layers

32. © Copyright Khronos Group, 2002 - Page 32 The Power of a Standard API To enable hardware and software communities But there are no acceptable cross-platform embedded graphics APIs -Particularly with advanced graphics capabilities We are missing this fundamental prerequisite for market growth -The graphics hardware community should take the imitative in driving the platform forward -Just as on the PC Standard Graphics API Embedded Hardware Community Embedded Software Community Requirements Capabilities Opportunities to optimize cost, power and performance for standardized functionality Can use advanced graphics capabilities consistently across multiple platforms

33. © Copyright Khronos Group, 2002 - Page 33 Cross Platform Solution Essential Many platforms and initiatives need these APIs WindML ® – expanding multimedia capabilities of VxWorks ® Looking to further expand 3D and rich media delivery capabilities Embedded Linux – need for low-footprint APIs No standardized 2D API other than X Windows – not low footprint Increasing use of OpenGL for 3D – but not low footprint Java Community – Java2D, Java Media Framework, Java Advanced Imaging Java3D is a high-level API – need low-footprint 3D API for embedded applications ARINC – standards for advanced cockpit displays Need standardized 2D & 3D APIs Severe certification demands drive need for minimum API size Leading Symbian OS for mobile consumer devices Needs minimal API size and increasing functionality Web3D Consortium Extensible standards for 3D delivery over the internet, driving 3D in MPEG-4 BUT needs target small footprint 2D/3D API for higher-level standards

34. © Copyright Khronos Group, 2002 - Page 34 OpenGL ES A proposed solution for advanced embedded graphics Create well-defined subset profiles of OpenGL -A series of increasingly compact profiles -Selecting functionality needed for key target markets Proper subsets ensure synergy with, and migration path to, full OpenGL -No new technologies are needed – just agreement on subsets of an existing API Eliminate redundancy Duplicated functionality Unnecessary queries Data-type variants Eliminate workstation functionality e.g. 3D textures Difficult/costly to implement with little benefit in embedded space Select functionality needed for key target markets

35. © Copyright Khronos Group, 2002 - Page 35 Poorly defined standard API requirements. Discussions with BroadcastCL group- to avoid divergence Strong support from ARINC and aerospace companies including Boeing, Airbus, Rockwell, Honeywell, Smiths, Lockheed… At a meeting in April 2002, Symbian and the cell phone industry selected Khronos’ “Embedded OpenGL” as the graphics API for cell phones with accelerated 3D in 2004 Phones Web pads Cameras PDAs Hand-held Appliances Line-powered Appliances STBs Internet Terminals Consoles Industry Momentum Safety Critical Avionics Automotive New Visual Processing APIs

36. © Copyright Khronos Group, 2002 - Page 36 Why OpenGL? The obvious starting point OpenGL is already the most widely available cross-platform graphics API -Workstations to embedded appliances, PCs to Macs, Windows to Linux Open Standard – not controlled by any one company -Specification controlled by the OpenGL Architecture Review Board (ARB) Low-level API -Enables lightest weight interface between software and hardware Well-proven specification of advanced graphics techniques -No need to re-invent the wheel Successfully implemented by many graphics companies -Low-cost chips and cores, software-only, floating point, fixed point Leveraged development -An ecology of development tools already based around OpenGL Strong future roadmap with OpenGL 2.0 -Enabling access to the new generation of programmable hardware Through Khronos – will be royalty free to the embedded industry -Khronos has built in a royalty free framework into its participation agreement -Agreed by SGI – MAJOR ACHIEVENT, MAJOR OPPORTUNITY

37. © Copyright Khronos Group, 2002 - Page 37 OpenGL ES Philosophy To enable rapid consensus Be strongly requirements-based -Handheld 3D gaming and vehicle displays are currently most urgent needs Small number of profiles -A large number will be come unwieldy Take large-scale profiling decisions – eliminate large functionality blocks -Don’t endlessly tweak individual function calls and parameters -It often makes little difference to size -Increases ISV confusion to have many detailed differences Enable software, fixed point implementations -As well as hardware accelerated implementations Define utility libraries to add additional functionality – e.g. display lists -Avoid burdening core profiles with functionality that can be implemented well in software

38. © Copyright Khronos Group, 2002 - Page 38 Some Profile Possibilities Addressing broad application requirements Based on 3Dlabs strawman -And early OpenGL ES ad hoc working group meetings Games 3D -Minimum footprint 3D - including texture-mapping -Good gaming platform Safety Critical 3D -Absolute minimum 3D for easy certification Core 2D -High-quality 2D geometry, Lines, polygons and text Core Image -High quality affine transforms Core 2D Core 3D Core Image Safety Critical 3D

39. © Copyright Khronos Group, 2002 - Page 39 Application Example Proposed profiles - solution for Avionics Displays Cockpit Display - typical requirements -Anti-aliased 2D geometric graphics -No windowing system -Minimum possible code size to minimize DO-178 certification costs -SOLUTION – Core 2D Enhanced raster requirements -Display of raster maps -Rotation required for “direction of flight up” mode -SOLUTION – Core 2D and Core Image Enhanced 3D required -For display of “out of window” 3D terrain -SOLUTION – Safety Critical 3D

40. © Copyright Khronos Group, 2002 - Page 40 Aggressive Timelines for OpenGL ES Embedded industry won’t wait Need ratifiable specs by the end of the year First ad hoc working group meetings at Siggraph July Khronos Participation agreement available here at Siggraph August Ad hoc Working Group Meetings 4Q02 First draft specification 1Q03 “OpenGL ES 1.0” ratified by Khronos

41. © Copyright Khronos Group, 2002 - Page 41 OpenGL 2.0 “Pure” OpenGL 2.0 OpenGL ES 2.0 How OpenGL Standards Fit Together OpenGL 1.3 OpenGL 1.x OpenGL 2.x “Pure” OpenGL 2.x OpenGL 2.x includes full 2.0 functionality – so all applications run unmodified OpenGL 1.3 GL2 extensions OpenGL 1.x GL2 extensions Applications can incrementally make use of new GL2 extensions OpenGL ES 1.0 Non-legacy applications can use recommended “Pure OpenGL” features and benefit from a simple, compact, efficient and flexible API

42. © Copyright Khronos Group, 2002 - Page 42 “embedded OpenGL” Scalable, small footprint 2D & 3D graphics “embedded OpenML” Small footprint video & audio processing SYNERGY Between Graphics and Media Processing Between Authoring & Deployment

43. © Copyright Khronos Group, 2002 - Page 43 Participation Framework Neil Trevett Vice President, 3Dlabs

44. © Copyright Khronos Group, 2002 - Page 44 New Participation Framework Khronos membership is open or interested companies There are four levels at which a company can participate -In OpenML and OpenGL® ES development Application developers -May use drivers implementing Khronos standards with no license or royalties Adopters -May sign a royalty free license to implement Khronos-based specifications Contributors -May apply for open membership in the Khronos Group at a nominal annual fee, with full working group participation and voting rights Promoters -Must be voted in by other Promoting Members and act as the “Board of Directors” to drive the direction of the Khronos Group and standardization activities, with final specification ratification voting rights.

45. © Copyright Khronos Group, 2002 - Page 45 How Khronos Membership Works Board of Promoters Promoters establish and participate in working groups for spec generation, marketing, reference implementations, conformance tests, etc. Contributor Membership Specification work undertaken in Working Group. Generates specs are passed back to the Promoters for final ratification Workin g Group Contributors participate in working groups and have full voting rights within those groups Ratified Khronos Group Specifications Ratified Khronos Group Specifications Application Developers Application Developers Adopters Adopters may sign a royalty free license to implement Khronos- based specifications Application developers may use drivers implementing Khronos standards with no license or royalties

46. © Copyright Khronos Group, 2002 - Page 46 Promoting Member Companies Dues of $15,000 per year Standards-Definition Benefits -Vote among Promoters to define objectives, priorities and operations of Khronos -Vote among Promoters on final ratification of specifications -Vote in the working groups Marketing Benefits -Company logo on trade show signage, and other marketing material -Quoted in Khronos Group press releases, whenever relevant -Speaking opportunities at major industry events on behalf of the Khronos Group -Opportunity to send staff to Khronos trade show booths and events -Logo and corporate description on the Khronos.org “Promoters” page -Listing the Khronos.org “Products” page

47. © Copyright Khronos Group, 2002 - Page 47 Contributing Member Companies Dues of $2,900 per year Standards-Definition Benefits -Vote in the working groups Marketing Benefits -Quoted in Khronos Group press releases, whenever relevant -Opportunity to send staff to Khronos trade show booths and events -Logo and corporate description on the Khronos.org “Contributors” page

48. © Copyright Khronos Group, 2002 - Page 48 Khronos IP Framework Shawn Underwood Marketing Director, SGI

49. © Copyright Khronos Group, 2002 - Page 49 Structure of Working Groups A working group per task OpenML 1.1 OpenGL ES 1.0 Implementation working groups Marketing working group Working Group Board of Promoters

50. © Copyright Khronos Group, 2002 - Page 50 The brave, new world of IP…. Khronos is taking a new approach to standards & IP Goal of Khronos is royalty-free specification Contributions to Khronos are royalty-free Members must actively participate Avoid hostage situations -Formal contributions -IP disclosure requirements Allow withdrawal to keep IP in-house

51. © Copyright Khronos Group, 2002 - Page 51 Highlights of Agreement With regards to IP Contributors only responsible for tracking activity in their working group Intellectual property is contributed formally Working group members must disclose IP that is impacted by recommendations to the board of Promoters Withdrawal from working group protects disclosed IP unless formally contributed Members may be compelled to contribute IP if it is not disclosed as such

52. © Copyright Khronos Group, 2002 - Page 52 Next Steps Neil Trevett Vice President, 3Dlabs

53. © Copyright Khronos Group, 2002 - Page 53 Participation Timeline Contributor Participation Agreements Available on web-site. 1-1 Discussions with Potential Promoters Board of promoters formed. First Promoters Meeting. Working Groups formed Working groups meetings start – signed up Promoters and Contributors can participate Siggraph 2002 September 6th

54. © Copyright Khronos Group, 2002 - Page 54 Next Steps Go to www.khronos.org To become a Contributor -Download and review Contributor Agreement from www.khronos.org -Execute and send, with dues, to Khronos to become a member If you want to be considered as a potential Promoter -Talk to Neil, Shawn or Elizabeth Working Groups activities on schedules on the web and mailing lists -We will inform you of first working group meetings Any questions – email the Khronos Manager -elizabeth@eriegelconsulting.com www.khronos.org

55. © Copyright Khronos Group, 2002 - Page 55 The Khronos Group encourages your participation to help grow the dynamic media market! Summary Khronos, OpenML and OpenGL ES Khronos has delivered its first specification - OpenML 1.0 -On schedule from one year ago at NAB 2000 OpenML is to dynamic media as OpenGL is to 3D graphics -Will reduce costs and increase productivity for anyone creating rich media Khronos is now commencing work on OpenML 1.1 -Further refinements, performance benchmarks and conformance tests Khronos is initiating OpenGL ES initiative -Produce profiled versions of OpenGL for embedded appliances and devices Strong synergy between authoring and playback environments -Call for involvement by any interested companies

56. © Copyright Khronos Group, 2002 - Page 56 Any Questions? IRIX and OpenGL are registered trademarks of Silicon Graphics, Inc., and OpenML is a trademark of Silicon Graphics, Inc., used with permission by the Khronos Special Interest Group. Sun, and Solaris are trademarks or registered trademarks of Sun Microsystems Inc. in the United States and other countries. Windows is a trademark of Microsoft Corp. Discreet is a trademark of Autodesk, Inc. / Discreet Logic. All other product names, trademarks, and/or company names are used solely for identification and belong to their respective owners.