Better Tools for Sound Designers on PlayStation®3 through Open Architecture Designs Michael Kelly, Senior Audio Engineer, Sony Computer Entertainment Europe

What does that mean?  On PlayStation 3  Very powerful audio engine  No tools to fully exploit it  Open architecture  Documented  Pluggable  Open Source  Published file format

Overview  Introduce PS3 Audio Engine  Reasons for rethinking audio tools  Discuss our approach  Show  How this model benefits game-audio

MultiStream TM Introduction  The de-facto PS3 Audio Engine  Entirely software based  Therefore it’s optional  Probably the most powerful and flexible audio engine in the world!

MultiStream Routing: Example: Voices to master  Up to 512 voices MASTER Voice 0 Voice n Voice 511

MultiStream Routing: Example: All busses to Master  Multiple busses to master MASTER Voice Bus 0 Bus n Bus 30 Voice

MultiStream Routing: Example: Busses route to each other  Asymmetrical configuration MASTER Bus 0 Bus 1 Bus 2 Bus 3 Voices Bus 4

MultiStream Effects  Compressor, Multiband Compressor, Softclip Distortion, Polynomial Distortion  Eq on bus, Parametric eq, Filters (15 types)  FDN reverb, I3DL2, Convolution Reverb  Pitch Shift/Time Stretch, Vocoder  Delay, Ring Modulator

MultiStream Effects  Each bus can have 8 effects Bus Slot 0 Filter Slot1 Reverb Slot2 Comp Slot3 EQ Slot4 Empty Slot5 Empty Slot6 Empty Slot7 Empty Voice

MultiStream Effects  …So can each voice Slot0: Stretch Slot1 Conv. Slot2 Empty Slot3 Empty Slot4 Empty Slot5 Empty Slot6 Empty Slot7 Empty

MultiStream Effects  Send and return Voices and busses Bus 0 Slot 0 Filter Slot1 Reverb Slot3 EQ Slot4 Empty Slot5 Empty Slot6 Empty Slot7 Empty Voices Bus 1 Slot 1 Filter Slot2 Reverb Slot3 EQ Slot4 Empty Slot5 Empty Slot6 Empty Slot7 Empty Voices Slot2 Send Slot 0 Return

MultiStream Routing: Send and return  Complicates routing further MASTER Bus 0 Bus 1 Bus 2 Bus 3 Voices Bus 4

MultiStream Multi-Channel  Each voice and bus  8 Channels (eg. 7.1 Surround)  Matrix Routing  Voice to Bus  Bus to Bus  Send to Return  Sends and returns  From/To any channel

MutliStream Power  We can run 50 X 2 second impulses  on a single SPU  Decode 400+ MP3 Streams  24kHz / 128kb/s

MultiStream Summary  Endlessly configurable  Many effect combinations  ~15 effects out of the box  ATRAC, VAG, MP3 support  Roll your own Effects  Roll your own formats  How do you make a tool to do that?

New Tool Design

No MultiStream Tools: What about SCREAM?

SCREAM / MultiStream  Uses fixed config MultiStream Bus Stream 0 SRC/ Pitch PEQDist.SendFilt Stream n SRC/ Pitch PEQDist.SendFilt Stream 127 SRC/ Pitch PEQDist.SendFilt

SCREAM Summary  Only uses a fraction of MultiStream  Specific to PlayStation platforms  Robust but dated

New Tool Spec  Easy to use  Provide complex routing and effects configuration  Improve workflow  Cross-Platform Engine  PS3, PlayStation 2, PSP  Xbox 360, PC, MAC

Easy to use  Basic assumption  -A whole other talk

Routing configuration  Sound Punk Tool (working title)  Desribe bus routing  Desribe effects config  Simplifies MultiStream setup

Cross Platform  Many approaches  Eg. Use Sound Punk to describe/emulate another platform  Configure MultiStream to be another engine  Lowest common denominator problem  Cross platform engines  Cross platform tools  What’s the difference

Cross Platform: The traditional approach  Middleware engine  Wraps console hardware  Fixed tools Hardware Engine Tool/Audio API Programmer

Cross Platform: Lowest common denominator  In the hardware days Platform API Tool

Cross Platform: The software engine  XBox360 and PS3  Very little audio hardware  Software engine  Same code  Except for optimizations

Cross Platform: The Tool Engine  With each tool you need the runtime engine  Bound to audio engine  Solves the lowest common denominator problem  This gives us a problem  MultiStream is optional!

Cross Platform: Separate tool & Engine  This is the way it used to be (kind of)  Good for us  Our tool works with other people’s engines  Good for other people because their tools work on our engine  Sound designer and programmer separate choice  It’s not that simple  & How to avoid the LCD problem

Separating the tool from the engine

Engine Overview  Abstract the engine  Configuration  Triggers  Param updates

Abstraction  Abstraction model: SP-EAL Tool API Engine Abstraction Engine

Sound Punk: Describe engine & config  Describes the audio engine  Max voices  Max busses  Effects  Routing Options  Describes specific configuration  Bus routing  Active effects

Abstraction + Sound Punk  Platform file describes engine limits  Voices, busses, effects, etc  Also describes config  bus names, source names, formats  Testing with different well-known APIs

Sound Punk: Supplied libraries on PS3  PS3 runtime library layer  Including source code  Developer can re-implement for other platforms (Wii, Xbox 360)  Example PC implementation  Script Engine also as source  Developer can rebuild for other platforms

Lowest common denominator: solved  Everything is abstracted  Mapping  Scalability

Engine abstraction: Bonuses  Once you abstract the engine, new things for free  Remote auditioning  In game preview  Cross platform

Problems  Voice allocation  Tool or engine  Missing effects  Different effects  File Formats  All implementation dependent  -Accommodate in tool

Another approach: The file format

Common File format  Engine abstraction and File Format  Not mutually exclusive  Agreed tool-level interchange format  Standard format for asset data  Standard format for scripts  Standard format for parameters

Tool choice Tool API Engine

Common file format  Advantages  Sound designer chooses right tool for right job  Common terminology  Common expectations  Should still allow USPs

Common File Format: Where to start?  Don’t reinvent the wheel  IASIG: iXMF  Interactive Extensible Media Format

Common File Format: XMF Folder layout File 1 File 2 Data Folder Nod e Folder Nod e Folder Nod e Data Nod e

Common File Format: Asset Types  Media files  + Chunks: references to media files  Cues and Scripts

Common File Format: Scripting  Script language defined  Roll your own  Everything is extensible

Common File Format: Extensibilty  Metadata  New script languages  New anything

Problems  GUI Metadata  Common file format, common tool?  Unsupported extensions?  Corruption  Inefficient at runtime?

Summary  Rationalising the sound process  SCEE has two approaches  Engine Abstraction + Config Tool  Open File Format  Perfect approach for us  What about you?  Improves workflow

Industry benefits  Mature idea of audio engine  Common terminology  True choice for developers  Improved Workflow  Better sounding games

This isn’t the end  (well it is, but..)  More discussion to follow  Sony Computer Entertainment & Developers  Events like GDC  Bodies like IASIG, AES, BBQ  And the Q&A…