1. WMP Overview
3/25/2017
Khronos Overview
Neil Trevett Vice President Mobile Content, NVIDIA President, Khronos Group
Copyright 2005, All rights reserved

2. Pervasive Mobile Media Computing
WMP Overview
3/25/2017
Pervasive Mobile Media Computing
Handsets are becoming personal computing devices - not “just” phones
Your most personal computer – mobility, connectedness and numerous sensors
Sophisticated media processing will be central to handheld revolution
Graphics and media will become as pervasive as it is on the PC
Diverse applications will drive the need for handheld accelerated media
User Interface, 2D maps and 3D navigation, TV/Video, games …
100Ms / year
10Ms / year
100Ks / year
Made text bigger – too hard to see from back of room
Media Platforms
80’s Workstations
90’s PCs
00’s Handhelds
Copyright 2005, All rights reserved

3. Khronos Open Standards
WMP Overview
3/25/2017
Khronos Open Standards
Khronos develops state-of-the-art open standards for the Authoring and Acceleration of graphics and media content
3D and Vector 2D Graphics, Video, Imaging, Audio
Khronos API standards provide “Foundation-Level” acceleration
“Close-to-the-silicon” access to media processors for efficiency and generality
Khronos standards are royalty-free
For wide adoption on multiple platforms – reducing application source fragmentation
Khronos APIs enable mobile software to be PORTABLE and to EFFICIENTLY use graphics and media acceleration
Copyright 2005, All rights reserved

4. WMP Overview
3/25/2017
Around 100 companies creating media authoring and acceleration standards
Copyright 2005, All rights reserved

5. Khronos Participation Model
Any company can join Khronos to participate in any number of working groups to produce specifications.
$7,500 annual membership dues
Promoters
Board decides strategy – approves working groups, controls budget, ratifies specifications.
$25,000 annual membership dues
Contributors
A Working Group for
each API standard – one company one vote
Conformance Tests and Conformance Test Process. Typically $10K per API fee
Openly and publicly distributed – free of charge, royalty free
Ratified Specifications
Conformance Tests
SDKs
Free libraries, utilities, examples, open source
Adopters
Conforming products can use API trademark and logo

6. Khronos Standards Ecosystem
Dynamic Media Authoring Standards
Dynamic Media Authoring
3D Authoring
Bridging 3D Authoring and Acceleration for advanced visual effects
Open Standards Effects Framework
Family of Market-focused 3D APIs
Vector 2D
Safety Critical 3D
Cross platform desktop 3D
Embedded 3D AL
Application Acceleration APIs
Streaming Media
Enhanced Audio
Embedded Media APIs
“DirectX-like” API set with OS Abstraction and API Interoperability
System Integration
Standards DL IL
CODEC and media component portability
Streaming Media System Integration
Open Standards for window systems

7. Adoption of Embedded Khronos APIs
WMP Overview
3/25/2017
Adoption of Embedded Khronos APIs
We Are Here
- OpenGL ES 1.1 is widespread
- OpenVG is in adoption ramp
- OpenMAX is in adoption ramp
Market adoption in media-accelerated handsets
100%
3D Graphics
Vector 2D
Enhanced Audio
Streaming Media
Mid-2004 OpenGL ES 1.1 Spec release
Mid-2005 OpenVG 1.0 Spec release
Beginning-2006 OpenMax IL 1.0 Spec release
THIS MONTH OpenSL ES 1.0 and OpenMAX AL 1.0 Spec release
News Flash!
Copyright 2005, All rights reserved

8. Penetration of Native Media APIs
WMP Overview
3/25/2017
Penetration of Native Media APIs
Copyright 2005, All rights reserved

9. OpenGL ES Applications and Devices
WMP Overview
3/25/2017
OpenGL ES Applications and Devices
OpenGL ES 1.1 has become the most widely deployed 3D API
Used in diverse applications, devices and markets
Advanced User Interfaces
iPhone uses OpenGL ES
3D Navigation
Google Earth and 3D city data
Mobile Games
$7.2Bn market in 2010 (Informa)
Nokia N93, iPhone, Sony PS3
All use OpenGL ES 1.1
Copyright 2005, All rights reserved

10. Complete Khronos Media Stack
WMP Overview
3/25/2017
Complete Khronos Media Stack
The Khronos API family provides a complete ROYALTY-FREE, cross-platform media acceleration platform
Applications or middleware libraries (JSR 184 engines, Flash players, media players etc.) AL
Playback and recording interfaces
Platform Media Frameworks
SOUND
Enhanced audio
3D Small footprint 3D for embedded systems
Vector 2D
Low-level vector acceleration API
Component interfaces for codec integration IL
EGL
Graphics surface management for efficient mixed mode 2D/3D/video rendering
Image Libraries, Video Codecs, Sound Libraries
Accelerated media primitives for codec development DL
Media Engines – CPUs, DSP, Hardware Accelerators etc.
Khronos defines low-level, FOUNDATION-level APIs.
“Close to the hardware” abstraction provides portability AND flexibility
Copyright 2005, All rights reserved

11. SOURCE PORTABILITY for rich media applications
A SET of native APIs to de-fragment mobile application development. Like DirectX – but an open standard for mobile platforms
Operating System Abstraction APIs Define APIs to isolate applications from operating system differences
SOURCE PORTABILITY for rich media applications
Media APIs with defined and tested Interoperability Include Khronos media APIs and define how they work together

12. TV/Video/Audio Players OpenKODE Core OS Abstraction
OpenKODE API Set
Software Platform (e.g. Brew, Symbian UIQ, S60, WIPI)
Java JVM
Native
Applications
Java
Applications
User Interface
Native Applications
3D Game Engines
Flash/SVG Players
TV/Video/Audio Players
Bindings
OpenKODE Core OS Abstraction 3D
Vector 2D
Streaming Media AL
Enhanced Audio
EGL Surface abstraction and Trans-API Communication
Media Acceleration
GPU / DSPs / CPUs
Kernel Operating System (e.g. Rex, Symbian OS, Windows Mobile, Linux, Nucleus)

13. OpenKODE Components
A conformant OpenKODE must pass all individual conformance tests PLUS trans-API tests to ensure APIs can communicate and operate concurrently
Trans-API Conformance Tests 3D
Vector 2D
EGL
Trans-API Communication
Selected Media APIs PLUS
any EGL necessary for those APIs to communicate.
(OpenGL ES 2.0, OpenMAX AL 1.0 and OpenSL ES 1.0 will be in OpenKODE 1.1 – added through extensions to OpenKODE 1.0)
Enhanced
Audio
Streaming Media
OpenKODE Core Events, IO, files, memory, threads, strings, math, locale, time, networking, windowing, crypto, debug
POSIX-like APIs - mandatory for a compliant OpenKODE. Standalone conformance tests
OpenKODE provides RELIABLE foundation-level acceleration for media applications that MIX multiple media types

14. SoftBank Mobile Announcement
New-generation mobile platform will provide native access to OpenKODE
POP-I – Portable Open Platform Initiative
Mandated native API set for advanced UI and games

15. OpenKODE Milestones OpenKODE 1.0 Provisional released at 3GSM 2007
WMP Overview
3/25/2017
OpenKODE Milestones
OpenKODE 1.0 Provisional released at 3GSM 2007
Twelve months from kick-off meeting to specification on web-site
Encouraging developer feedback before spec finalization
Full conformance tests being prepared in parallel with specification
Written by Futuremark – tests OpenKODE Core AND trans-API operation
OpenKODE 1.0 Final Release Target with full conformance tests
OpenKODE 1.1?
Release – add OpenMAX and OpenSL ES support
OpenKODE 1.0 Provisional Release
1Q07
2Q07
3Q07
4Q07
1Q08
2Q08
3Q08
Provisional Conformance Tests to be Released
Copyright 2005, All rights reserved

16. OpenKODE 1.1 - 3D Video Textures
WMP Overview
3/25/2017
OpenKODE D Video Textures
EGL is a Khronos API for abstracting rendering surfaces
OpenKODE 1.1 working to define how surfaces shared between OpenGL ES and OpenMAX
Aiming to eliminate unnecessary copies for power and processing efficiency
Without EGLImage the application must copy
the OpenMAX buffer into an OpenGL ES texture
Data copy wastes cycles and space
MPEG Video
OpenMAX MPEG-4 Decode
OpenMAX Buffer
glTeximage2D Surface
OpenGL ES 3D Engine
An EGLImage surface can be used as both the destination of the decode and a texture without copying the data
MPEG Video
OpenMAX MPEG-4 Decode
EGLImage Surface
OpenGL ES 3D Engine
Copyright 2005, All rights reserved

17. Raising 2D and 3D Visual Quality
WMP Overview
3/25/2017
Raising 2D and 3D Visual Quality
State-of-the-art APIs enable compelling consumer displays
Advanced functionality, fast interactivity and extremely high quality
High-quality 2D graphics using OpenVG
High-quality 3D displays using OpenGL ES
Video processing with OpenMAX
Older generation APIs
Provide rudimentary graphics functionality and quality
Copyright 2005, All rights reserved

18. COLLADA = Powerful Authoring
COLLADA is a XML database schema for 3D assets
Can hold geometry, animation, visual effects, physics – everything to do with a scene
COLLADA transports 3D assets between applications
Enables binding of diverse DCC and 3D processing tools into a production pipeline
COLLADA can be lossless – never lose information
Retains all information - even multiple versions of the same asset
COLLADA is an open, archive-grade format that retains meta information
When your DCC tool upgrades, you keep your assets
COLLADA is non-destructive and so supports round-tripping of tools to enable powerful authoring pipelines
Custom Tools

19. WMP Overview
3/25/2017
COLLADA Momentum
COLLADA 1.4 supported by all major tools and thousands of users
3ds Max, Photoshop CS3, Blender, DAZ|Studio, C4 Game Engine, Poser
NVIDIA FX Composer, Google Earth, Houdini, Maya, Sketchup, and XSI
DAZ|Studio 1.7 exports COLLADA
Complete export of mesh, skinning, rigging, lighting, camera, and animation data.
Caligari trueSpace7.5 exports COLLADA
Adobe Photoshop CS3 Extended imports COLLADA
For texture editing on 3D objects
Copyright 2005, All rights reserved

20. COLLADA UI Bringing 2D vector graphics to COLLADA file format
Enables deep intermixing of 2D and 3D graphics
Support for 2D and 3D Font libraries
Font glyphs to support effects and animation for advanced rendering

21. COLLADA Conditioning
Conditioning pipelines take authored assets and:
1. Strips out authoring-only information
2. Re-sizes to suit the target platform
3. Compresses and formats binary data for the target platform
Different target platforms can use the same asset database with the appropriate conditioning pipeline
COLLADA is an interchange format - not a delivery format or a scene graph
Multiple tools create assets and scenes in a COLLADA Database
Conditioning Pipeline
Conditioning Pipeline

22. glFX – Effects Framework
WMP Overview
3/25/2017
glFX – Effects Framework
COLLADA
File format for 3D asset interchange – widely adopted by DCC tools vendors and Google, Adobe, Epic etc.
- Existing Khronos standards
- New glFX standard
COLLADA FX
Textures, shaders programs, geometry, control and pass information
Conditioning
Runtime file format
glFX File Format - OPTIONAL
Contains effects optimized for deployment
1) The DCC application creates COLLADA FX ES 2 profile content (geometry, animation, material, effect, lights) and shaders. This can be a modeling package, level editor, some sort of YSIWYG rendering tool.
2) Once COLLADA content (reference to image data / materials which associate effects with parameters / instances of materials on geometry / lights / scene hierarchy / skinning information / shaders, effects, techniques or referenced to an external file (CgFx or MS FX) ) is created, it is fed into the conditioner pipeline. Here, the content is made more HW friendly and compressed. Might create indexed geometry, tri-strips. Compress textures, filter out duplicate shaders, etc. The result is the scene binary, which contains all data needed for the rendering (game) engine at deployment time, and the FX ES binary containing shaders, in binary form, or portable form, multi-pass information, techniques, render state information etc.
3) The game engine traverses its optimized scene data, uses the FX ES runtime API to extract what it needs out of the FX ES binary. It can also use the runtime to test techniques in the binary depending on the underlying hw capabilities.
4) The FX ES binary should allow for storing different techniques for the same effect depending on HW capabilities. Maybe also can express the use of OpenGL ES extensions as a set of techniques.
Application
Runtime API
glFX Run-time API
Application traverses scene data, uses the glFX Runtime API to use effects information to setup the rendering pipeline
OpenGL 2.1 / OpenGL 3.0 / ES (Also ES 1.1 / Mount Evans)
Copyright 2005, All rights reserved

23. Benefits of One Organization
OpenGL and OpenGL ES are now evolving under one IP framework
Design innovations can be freely shared between the APIs
OpenGL and OpenGL ES can share same resources and outreach
Common Conformance tests, marketing and web-site, tool chains etc.
OpenGL and OpenGL ES Working Groups will remain independent
Both groups will be able to make decisions that best serve their own markets
Architectural design expertise
Embedded Markets
Desktop Markets
Market feedback on streamlining functionality
Momentum - hundreds of millions of OpenGL ES devices

24. OpenGL Roadmap Synergy
OpenGL ES has leveraged desktop OpenGL architecture
OpenGL ES 1.1 streamlined OpenGL 1.5
OpenGL ES 2.0 streamlined OpenGL 2.0 and the GLSL shading language
Next Generation OpenGL 3 and “Mount Evans” leverages OpenGL ES 2.0
Learning from the streamlining of OpenGL ES 2.0 core
OpenGL ES will benefit from the architectural innovation of “Mount Evans”
Nexgen OpenGL ES – “Halti” will use desktop innovations for performance and functionality
“Halti” Next generation functionality for embedded markets
OpenGL ES 1.1/2.0 Functional Streamlining
OpenGL 1.5/2.0 Architectural Foundation
“OpenGL 3 / Mount Evans” Architectural Streamlining and next generation functionality

25. OpenGL 3 and Mount Evans OpenGL 3 – before end of 2007
Architectural streamlining of OpenGL
Elimination of fixed functionality – evolving work in OpenGL ES 2.0
New object model for speed and simpler programming
GL allocates handles, Direct editing, Efficient state grouping, Efficient sharing
OpenGL 3 provides backwards compatibility
OpenGL 3 requires a new context
Both OpenGL 2.1 and OpenGL 3 contexts may bind to same drawable
Legacy code and LP code may peacefully co-exist - developers migrate at their own pace
Mount Evans – early 2008
Adds significant new features to OpenGL 3
Geometry shaders
Greatly increased integer support in the OpenGL Shading Language
Stream out of vertex data to a buffer object
Texture arrays
Texture buffer objects
Numerous new texture formats
Instanced rendering

26. Khronos 2007 Milestones OpenMAX AL 1.0 Provisional Released
OpenSL ES 1.0 Provisional Released
OpenVG 1.1 Public Release
OpenKODE 1.0 Final Release
COLLADA Conformance Tests Release
OpenGL 3 Release
4Q07
iPhone launched – uses OpenGL ES 1.1 for UI and display composition
COLLADA Adopted by Adobe for Photoshop CS3
OpenVG 1.0 Conformance Tests Released
OpenVG 1.0 Reference Implementation open sourced
2Q07
glFX Working Group Announced
OpenWF Working Group Announced
COLLADA UI TSG Announced
OpenGL 3 features revealed
3Q07
OpenMAX IL 1.1 Public Release
with Conformance Tests
OpenKODE 1.0 Provisional Released
1Q07

27. Why Are Khronos Standards Key?
“Foundation Level” APIs to enable software to effectively use silicon
Close to the silicon – fundamental functionality needed on every platform
Synergistic mix of authoring and acceleration standards
Being designed together
Hundreds of man years invested
Beyond any single company now to produce specifications of this breadth and depth
Royalty-free
Khronos is committed to generating market opportunities for its members and the industry

28. Help Khronos Help You! Please consider joining Khronos!
Developer feedback and participation is always very welcome
Please review our specifications and provide feedback
Public forums and developer resources here
Slides and Khronos membership details at