Source Multicore 1 November 2006

New Source Features Dynamic Shadow Mapping New Foliage System Cinematic Physics Version 2 of Facial Animation System Dynamic Scripted Sequences Particle System 64-Bit Source Engine Reworked Character Lighting Model Companion AI (Alyx in Episode One) Real-Time statistics gathering Automatic game cache defragmentation Multicore

Multiple Approches to multi-core games New Features in Source Multiple Approches to multi-core games Source is implementing Hybrid-Threading Close to linear improvements Pretty but dumb era is ending, scalability we've seen in graphics will now apply to rest of the game Hybrid Threading Source will ship before Episode Two You'll get a version of this engine to take home to test with

Most significant development since 3D cards Multicore Most significant development since 3D cards Huge potential Huge challenge

Most significant development since 3D cards Multicore Most significant development since 3D cards Huge potential Huge challenge The decisions faced with multiple cores How we are using multiple cores

Most significant development since 3D cards Multicore Most significant development since 3D cards Huge potential Huge challenge The decisions faced with multiple cores How we are using multiple cores Four cores is more than twice as interesting as two cores

Challenges

Games always want 100% CPU utilization Challenges Games always want 100% CPU utilization

Games always want 100% CPU utilization Games are inherently serial Challenges Games always want 100% CPU utilization Games are inherently serial

Games always want 100% CPU utilization Games are inherently serial Challenges Games always want 100% CPU utilization Games are inherently serial Decades of experience in single threaded optimization

Games always want 100% CPU utilization Games are inherently serial Challenges Games always want 100% CPU utilization Games are inherently serial Decades of experience in single threaded optimization Millions of lines of code written for single threading

Strategies Threading model Threading framework Application of cores

Fine grained threading Hybrid threading Theading Models Single threading Coarse threading Fine grained threading Hybrid threading

Single threading Easy Obsolete

Put whole systems on cores Pretty easy, “multiple single threads” Coarse threading Put whole systems on cores Pretty easy, “multiple single threads” Stay partially serialized, or double buffer

Coarse threading: Early experimentation Client User input Rendering Graphics simulation Server AI Physics Game logic

Coarse threading: Early experimentation Experiment: run client and server each on own core

Coarse threading: Early experimentation Experiment: run client and server each on own core Benefits: forced to confront systems that are not thread safe or not thread efficient

Coarse threading: Early experimentation Experiment: run client and server each on own core Benefits: forced to confront systems that are not thread safe or not thread efficient Outcome: Can approach 2x in contrived maps

Coarse threading: Early experimentation

Coarse threading: Early experimentation

Coarse threading: Early experimentation Experiment: run client and server each on own core Benefits: forced to confront systems that are not thread safe or not thread efficient Outcome: Can approach 2x in contrived maps More like 1.2x in real single player Added latency to single player game

Coarse threading: Early experimentation Experiment: run client and server each on own core Benefits: forced to confront systems that are not thread safe or not thread efficient Outcome: Can approach 2x in contrived maps More like 1.2x in real single player Added latency to single player game Opened door to improved listen servers

Put whole systems on cores Pretty easy, “multiple single threads” Coarse threading Put whole systems on cores Pretty easy, “multiple single threads” Stay partially serialized, or double buffer Scales poorly Partially idle cores Synchronization, or lag Entirely idle cores

Fine grained threading Divide many small identical tasks across cores E.g., take a loop that updates state of 1000 objects and perform 1000/N on each core for N cores Moderate difficulty Scales well… Tricky if cost of each unit is variable Memory bandwidth Limited problem domains

Fine grained threading Leverage multicore in production tools: VMPI

Fine grained threading VVIS – Visibility calculations VRAD – Lighting calculations

Fine grained threading

Performance tuned for mid-level work sharing Hybrid threading Performance tuned for mid-level work sharing Not splitting sets of very small operations over cores Not putting whole systems onto cores

Use the appropriate tool for the job Hybrid threading Use the appropriate tool for the job Some systems on cores (e.g. sound) Some systems split internally similar to coarse Split expensive iterations across cores fine grained Queue some work to run when a core goes idle Most difficult Scales well Maximum core utilization

Hybrid threading: Rendering Rough pipeline Build world lists Build object lists Graphical simulation (particles, ropes, sprites) Update animations Compute shadows Draw Once for every “view” Player’s POV TV monitors Water reflections Many times CPU bound

Hybrid threading: Rendering Revised pipeline Construct scene rendering lists for multiple scenes in parallel (e.g., the world and its reflection in water) Overlap graphics simulation Compute character bone transformations for all characters in all scenes in parallel Compute shadows for all characters Allow multiple threads to draw in parallel Serialize drawing operations on another core

Implementing Hybrid Threading Operating system: pools, synchronization Threading Tools Implementing Hybrid Threading Operating system: pools, synchronization Compiler extensions: OpenMP, fine threading Tailored tools Programmers solve game development problems, not threading problems

Operating system Too low level Prone to error Lots of stalling Unpredictable scheduling Unpredictable cost

Focused on fine threading Lack of control Implementation interferes Compiler extensions OpenMP Focused on fine threading Lack of control Implementation interferes

Tailored tools: Game Threading Infrastructure Custom work management system Aimed at gaming problems, intuitive for game programmers Focus on keeping cores busy Thread pool: N-1 threads for N cores Support hybrid threading Function threading Array parallelism Multiple work modes Opportunistic core utilization Queued core utilization

Tailored tools: Game Threading Infrastructure The simple thing is the worst thing “Lock-free” algorithms Never leave cores idle waiting on other cores Leverages atomic write primitives of the CPU Under the hood of all services and data structures See: http://en.wikipedia.org/wiki/Lock-free_and_wait- free_algorithms Example: the spatial partition

Application of cores

Dual core CPUs: framerate Application of cores Dual core CPUs: framerate

Dual core CPUs: framerate Quad core CPUs: new experiences Application of cores Dual core CPUs: framerate Quad core CPUs: new experiences

Dual core CPUs: framerate Quad core CPUs: new experiences Application of cores Dual core CPUs: framerate Quad core CPUs: new experiences Richer visuals Improved simulation Richer AI

Application of cores: Particle Simulation

Application of cores: Particle Simulation Not simply a GPU issue Interactivity Presence

Application of cores: Particle Simulation Use cores to run multiple particle systems in parallel Individual particle systems using multiple cores

Application of cores: Particle Simulation More complicated systems Interactive particle systems Particles with gameplay implications Like rigid body physics, reinforce consistency of world

Particle Simulation Benchmark

Traditionally strict CPU limits AI Traditionally strict CPU limits Interesting combinations of minimalist algorithms “What could we do if extra CPU were given to AI?”

Better framerate AI Run AI in parallel with other systems Parallel agent execution

Example: Parallel Animation AI Example: Parallel Animation

Increased sophistication AI Better framerate Increased sophistication Without hitching by asynchronously running on secondary cores

Increased sophistication AI Better framerate Increased sophistication Without hitching by asynchronously running on secondary cores Richer path finding: better world interactivity

Increased sophistication AI Better framerate Increased sophistication Without hitching by asynchronously running on secondary cores Richer path finding: more analysis for better world awareness Deeper tactical analysis Increased world examination “Out of band” AI

Increased sophistication New kinds of AIs Better framerate Increased sophistication New kinds of AIs Experimental creatures built with multicore in mind

Integrate multicore across Valve’s business Broader Goals Integrate multicore across Valve’s business

Integrate multicore across Valve’s business Broader Goals Integrate multicore across Valve’s business Boost developer effectiveness

Integrate multicore across Valve’s business Broader Goals Integrate multicore across Valve’s business Boost developer effectiveness Expose to game programmers, licensees and MOD authors

Integrate multicore across Valve’s business Broader Goals Integrate multicore across Valve’s business Boost developer effectiveness Expose to game programmers, licensees and MOD authors Transparently scale to cores without recompile

Integrate multicore across Valve’s business Broader Goals Integrate multicore across Valve’s business Boost developer effectiveness Expose to game programmers, licensees and MOD authors Transparently scale to cores without recompile Leverage to 360

Integrate multicore across Valve’s business Broader Goals Integrate multicore across Valve’s business Boost developer effectiveness Expose to game programmers, licensees and MOD authors Transparently scale to cores without recompile Leverage to 360 Provide value to our customers beyond framerate

Integrate multicore across Valve’s business Broader Goals Integrate multicore across Valve’s business Boost developer effectiveness Expose to game programmers, licensees and MOD authors Transparently scale to cores without recompile Leverage to 360 Provide value to our customers beyond framerate Pre-Episode 2 Steam to deliver to customers, MOD authors, and licensees

Multicore is exciting and scary Accessible solutions Hybrid Threading Closing thoughts Multicore is exciting and scary Accessible solutions Hybrid Threading Scalable More broadly applicable Game Threading Infrastructure Closely controlled core usage Tools framed in terms of game problems Apply cores to new visual gameplay opportunities

Experiments and Benchmarks Particle System Multicore AI Provided to take home: VRAD benchmark, particle simulation benchmark