2. Better Tools for Sound Designers on PlayStation®3 through Open Architecture Designs Michael Kelly, Senior Audio Engineer, Sony Computer Entertainment Europe

3. What does that mean?  On PlayStation 3  Very powerful audio engine  No tools to fully exploit it  Open architecture  Documented  Pluggable  Open Source  Published file format

4. Overview  Introduce PS3 Audio Engine  Reasons for rethinking audio tools  Discuss our approach  Show  How this model benefits game-audio

5. MultiStream TM Introduction  The de-facto PS3 Audio Engine  Entirely software based  Therefore it’s optional  Probably the most powerful and flexible audio engine in the world!

6. MultiStream Routing: Example: Voices to master  Up to 512 voices MASTER Voice 0 Voice n Voice 511

7. MultiStream Routing: Example: All busses to Master  Multiple busses to master MASTER Voice Bus 0 Bus n Bus 30 Voice

8. MultiStream Routing: Example: Busses route to each other  Asymmetrical configuration MASTER Bus 0 Bus 1 Bus 2 Bus 3 Voices Bus 4

9. MultiStream Effects  Compressor, Multiband Compressor, Softclip Distortion, Polynomial Distortion  Eq on bus, Parametric eq, Filters (15 types)  FDN reverb, I3DL2, Convolution Reverb  Pitch Shift/Time Stretch, Vocoder  Delay, Ring Modulator

10. MultiStream Effects  Each bus can have 8 effects Bus Slot 0 Filter Slot1 Reverb Slot2 Comp Slot3 EQ Slot4 Empty Slot5 Empty Slot6 Empty Slot7 Empty Voice

11. MultiStream Effects  …So can each voice Slot0: Stretch Slot1 Conv. Slot2 Empty Slot3 Empty Slot4 Empty Slot5 Empty Slot6 Empty Slot7 Empty

12. MultiStream Effects  Send and return Voices and busses Bus 0 Slot 0 Filter Slot1 Reverb Slot3 EQ Slot4 Empty Slot5 Empty Slot6 Empty Slot7 Empty Voices Bus 1 Slot 1 Filter Slot2 Reverb Slot3 EQ Slot4 Empty Slot5 Empty Slot6 Empty Slot7 Empty Voices Slot2 Send Slot 0 Return

13. MultiStream Routing: Send and return  Complicates routing further MASTER Bus 0 Bus 1 Bus 2 Bus 3 Voices Bus 4

14. MultiStream Multi-Channel  Each voice and bus  8 Channels (eg. 7.1 Surround)  Matrix Routing  Voice to Bus  Bus to Bus  Send to Return  Sends and returns  From/To any channel

15. MutliStream Power  We can run 50 X 2 second impulses  on a single SPU  Decode 400+ MP3 Streams  24kHz / 128kb/s

16. MultiStream Summary  Endlessly configurable  Many effect combinations  ~15 effects out of the box  ATRAC, VAG, MP3 support  Roll your own Effects  Roll your own formats  How do you make a tool to do that?

17. New Tool Design

18. No MultiStream Tools: What about SCREAM?

19. SCREAM / MultiStream  Uses fixed config MultiStream Bus Stream 0 SRC/ Pitch PEQDist.SendFilt Stream n SRC/ Pitch PEQDist.SendFilt Stream 127 SRC/ Pitch PEQDist.SendFilt

20. SCREAM Summary  Only uses a fraction of MultiStream  Specific to PlayStation platforms  Robust but dated

21. New Tool Spec  Easy to use  Provide complex routing and effects configuration  Improve workflow  Cross-Platform Engine  PS3, PlayStation 2, PSP  Xbox 360, PC, MAC

22. Easy to use  Basic assumption  -A whole other talk

23. Routing configuration  Sound Punk Tool (working title)  Desribe bus routing  Desribe effects config  Simplifies MultiStream setup

24. Cross Platform  Many approaches  Eg. Use Sound Punk to describe/emulate another platform  Configure MultiStream to be another engine  Lowest common denominator problem  Cross platform engines  Cross platform tools  What’s the difference

25. Cross Platform: The traditional approach  Middleware engine  Wraps console hardware  Fixed tools Hardware Engine Tool/Audio API Programmer

26. Cross Platform: Lowest common denominator  In the hardware days Platform API Tool

27. Cross Platform: The software engine  XBox360 and PS3  Very little audio hardware  Software engine  Same code  Except for optimizations

28. Cross Platform: The Tool Engine  With each tool you need the runtime engine  Bound to audio engine  Solves the lowest common denominator problem  This gives us a problem  MultiStream is optional!

29. Cross Platform: Separate tool & Engine  This is the way it used to be (kind of)  Good for us  Our tool works with other people’s engines  Good for other people because their tools work on our engine  Sound designer and programmer separate choice  It’s not that simple  & How to avoid the LCD problem

30. Separating the tool from the engine

31. Engine Overview  Abstract the engine  Configuration  Triggers  Param updates

32. Abstraction  Abstraction model: SP-EAL Tool API Engine Abstraction Engine

33. Sound Punk: Describe engine & config  Describes the audio engine  Max voices  Max busses  Effects  Routing Options  Describes specific configuration  Bus routing  Active effects

34. Abstraction + Sound Punk  Platform file describes engine limits  Voices, busses, effects, etc  Also describes config  bus names, source names, formats  Testing with different well-known APIs

35. Sound Punk: Supplied libraries on PS3  PS3 runtime library layer  Including source code  Developer can re-implement for other platforms (Wii, Xbox 360)  Example PC implementation  Script Engine also as source  Developer can rebuild for other platforms

36. Lowest common denominator: solved  Everything is abstracted  Mapping  Scalability

37. Engine abstraction: Bonuses  Once you abstract the engine, new things for free  Remote auditioning  In game preview  Cross platform

38. Problems  Voice allocation  Tool or engine  Missing effects  Different effects  File Formats  All implementation dependent  -Accommodate in tool

39. Another approach: The file format

40. Common File format  Engine abstraction and File Format  Not mutually exclusive  Agreed tool-level interchange format  Standard format for asset data  Standard format for scripts  Standard format for parameters

41. Tool choice Tool API Engine

42. Common file format  Advantages  Sound designer chooses right tool for right job  Common terminology  Common expectations  Should still allow USPs

43. Common File Format: Where to start?  Don’t reinvent the wheel  IASIG: iXMF  Interactive Extensible Media Format

44. Common File Format: XMF Folder layout File 1 File 2 Data Folder Nod e Folder Nod e Folder Nod e Data Nod e

45. Common File Format: Asset Types  Media files  + Chunks: references to media files  Cues and Scripts

46. Common File Format: Scripting  Script language defined  Roll your own  Everything is extensible

47. Common File Format: Extensibilty  Metadata  New script languages  New anything

48. Problems  GUI Metadata  Common file format, common tool?  Unsupported extensions?  Corruption  Inefficient at runtime?

49. Summary  Rationalising the sound process  SCEE has two approaches  Engine Abstraction + Config Tool  Open File Format  Perfect approach for us  What about you?  Improves workflow

50. Industry benefits  Mature idea of audio engine  Common terminology  True choice for developers  Improved Workflow  Better sounding games

51. This isn’t the end  (well it is, but..)  More discussion to follow  Sony Computer Entertainment & Developers  Events like GDC  Bodies like IASIG, AES, BBQ  And the Q&A…