Multi-threading the Oxman Game Engine Sean Oxley CS 523, Fall 2012

Project Overview An engine is middleware for game developers, handling low-level details Adaptation of a single-threaded video game engine to multi-threaded

What's being multi-threaded? Physics Spatial queries, synchronization Animation Updating of time Keyframe retrieval, PRS generation Skeleton matrix palette generation Visibility Visibility AABB generation Spatial tree classification Camera culling pass

Challenges of multi-threaded game engines Response time At 60 frames per second (ideal framerate), one frame is ~16.7 ms Threads must compute short bursts of data in as little time as possible Synchronization and start-up costs must be minimized Transparency to developer Developer should not be expected to maintain strict access rules (preferably, none at all) Unpredictability of object behavior and data access

The approach Utilize data parallelism Pool of threads that all simultaneously work on a task Batch physics, animation, and visibility calculations into arrays High independence of work Synchronization code minimal, mostly contained in one class Theoretically scalable to larger numbers of threads Non-parallel code runs just as fast as before Little impact on developer

Caveats of the approach Third-party libraries Bullet Physics, what Oxman Engine currently uses for physics, is not thread-safe Intention was to do simultaneous queries, but forced to resort to background queries Large implications for CharacterController performance Slight developer discipline requirements Physics Waiting for queries to complete Maintaining data lifetime for the query's duration Availability Results of animation/physics not reflected until post physics/animation update phase

The details A “job” (a static function designed to be run in parallel) is launched by the application through a class called ThreadPool Jobs can be launched either blocking or non-blocking If blocking, the calling thread also participates in the job After the job is launched, the caller sleeps until the job is complete A job can utilize any number of threads, but only one job can run at a time

The details, cont. Threads are created a priori at engine init Avoids overhead of constant thread creation/destruction Threads are woken up, and busy-wait until all threads have woken up (avoids race conditions) Each thread runs the job function, then goes back to sleep The threads know their index and how many other threads are performing the job If the job was blocking, the last thread wakes the calling thread up

Issues from compiler optimization and out-of-order execution Busy-waiting on a variable to be a desired value Compiler will optimize out the load instruction Causes deadlock; must force a reload Out-of-order execution Could be done by either CPU or compiler Cannot rely on execution order without special instruction; potential for race conditions This instruction is technically not supported on all architectures, but will exist on the engine's target machines Both problems solved by the MemoryBarrier() macro on Win32 (macro for inline assembly)

Addressing performance concerns In first implementation, threads slept on a condition variable Worked fine with two threads, awful with any more Problems caused by both high kernel and high mutex contention (at least on Windows) Busy-waiting didn't work either OS was more likely to preempt, holding up the whole pool Compromise made Threads continually check for a job. If no job is there, they call Sleep(0), which gives up their time quantum Makes OS preemption much less likely

The Performance Test Test machines: Testing procedure: Intel Core i7 920 2.67 GHz (8 logical cores), 16 GB RAM, nVidia Geforce GTX 570 Pentium Dual T3400 2.16 GHz (2 logical cores), 3 GB RAM, Intel 4 Express (integrated) Testing procedure: First test: 10 characters, using CharacterController components More of a realistic scenario Second test: 200 characters, not using the CharacterController More of a stress test for the parallelized engine portions Both tests done once with rendering on, once with rendering off

Core i7 Results, Part 1

Core i7 Results, Part 2

Core i7 Performance Analysis First test results were rather inconclusive Differences in frame times could be attributed to error in measurements 8 threads actually slows down performance Second test results much more clear 1, 2, and 4 threads showed a 5 ms improvement each time 8 threads didn't provide any benefits

Pentium Dual Results, Part 1

Pentium Dual Results, Part 2

Pentium Dual Performance Analysis No improvement for the first test Benefit of two threads offset by costs of preemption Second test again fared much better A whopping 11-12 ms improvement

Conclusion As it currently stands, the engine's multi-threading has limited use Second test is a rather unlikely scenario Scalability concerns Fork-and-join might be more suited for consoles Not being able to effectively parallelize physics was a huge issue Other, more complex approaches will need to be tried Given more time, I'd try the job queue approach next

The roads not traveled Use of a thread-safe physics library Spatial queries could then be batched and executed all in parallel CPU-intensive CharacterControllers could be parallelized Different multi-threading approaches Threads dedicated to particular subsystems “Start-up time” issues reduced or eliminated Harder to synchronize, Scalability issues Smaller-granularity job queue Work is put in a queue and picked up by individual threads, instead of all threads at once Scales better as # of threads increase More flexible Difficulty of scalable job creation More game-like performance testing scenarios