A Mathematical Model for Balancing Co-Phase Effects in Simulated Multithreaded Systems Joshua L. Kihm, Tipp Moseley, and Dan Connors University of Colorado at Boulder

Exploiting Phase Behavior for Efficient Architecture Simulation Program behavior patterns, or Phases, can be exploited for efficient simulation [Simpoint-Sherwood, et al. PACT ’01] –Capture repeating phase and eliminate simulation time or direct detailed simulation Industry trends towards multithreaded processors –In a multithreaded system, execution is characterized by a combination of phases between co-resident threads, called a Co-Phase [VanBiesbrouk, et al., ISPASS ’04] –Phase exploitation more difficult for simulation and design of multithreaded systems since the individual phases interact in unique ways

Program Execution Terminology Period ACAAAAAAAAAABBBB BCCCCPhase Interval PERIOD – A segment of program execution of a given length (one OS scheduling period in this work) PHASE – A set of periods with similar behavior INTERVAL – A set of consecutive periods with the same phase (one occurrence of a phase)

Effects of Co-phases 181.mcf and 186.crafty Relative progress of threads is determined by individual thread behavior and inter-thread interference As Co-Phase changes, so does interference and performance Data from Pentium-4 (Northwood) system illustrates co-phase effects and transitions [Graph format from VanBiesbrouk] What if we start here? Or here?

Problem Statement Variation in offset between threads causes variation in which co-phases are encountered and their relative importance –>15% standard deviation in IPC for some combinations. Offset is caused by: –Start Times –OS Scheduling –Simulation Error The average performance must be determined in order to reflect real system performance where the relative position of threads will be randomized

Example Analysis of Pentium-4 Data Total ST runtime HT performs below ST! Best performance at high offset

Motivation (Methodology) Tested on implemented hardware –Intel Pentium-4 Northwood with Hyperthreading Used 5 SPEC CPU 2000 benchmarks –188.ammp, 179.art, 186.crafty, 252.eon, 181.mcf –Long-running benchmarks Offsets of –100s,-90s, -80s, … +80s, +90s, +100s (21 tests per pairing)

Performance Variance Due to Offset Percent standard deviation Variation is high for many metrics Self-pairings have high variation

Co-Phase Variance Difference in portion of time spent in each co-phase Co-phase mix changes with offset

Conceptual Model The time spent in each co-phase interval will determine overall performance The amount of time in the co-phase interval is dependent on each thread’s: –Performance in co-phase –Length of the interval –Number of operations already completed in the current interval of each thread

Determining the Time in Co- Phase Interval Interval length and co-phase performance are constant, but need to be determined ahead of time *Assumption of phase-based simulation The number of a operations already completed is a function of previous performance and co-phase profile

Determining the Time in a Co- Phase Interval Offset Time in Interval Interval i runs in its entirety Interval is not encountered Similar case for thread Y Part of Interval occurs (Monotonic) Overall case is the minimum Thread Y changes phase first Next interval is (i,j+1) Thread X changes phase first Next interval is (i+1,j) Area under the curve is proportional to average length of the interval

Mathematical Model Thread X finishes first Thread Y finishes first Performance in Co-Phase Number of operations in interval Number of operations yet to complete in interval Number of operations already completed in Interval

Start-up Intervals Interval lengths are dependent on previous intervals (the total number of retired operations) all the way back to the start of execution of the thread –Some model is needed to simulate the number of operations difference between thread Model based on single-threaded behavior *Assume that single phase behavior is indicative of average co-phase behavior

Deriving Start-up Intervals Offset Time in Interval Length of Phase Interval Interval is never entered Interval doesn’t occurCo-phase i,1Next Startup interval i+1,0 i+1,0 i-1,0 Interval length equals offset M=1 Length of phase i

Mathematical Model (Start-up Interval) Length of intervals up to “i” Length of intervals up to and including “i” Partial completion Interval not encountered Full completion

Example analysis of Pentium-4 Data First two intervals of 186.crafty and 188.art

Example Analysis of Pentium-4 Data Run time of crafty Run time of art Total ST runtime Art Phase 2 causes heavy interference HT performs below ST! Best performance at high offset

Extensions to More Threads One thread is “reference” –Arbitrarily chosen –Number of variables grows linearly Concepts and equations easily extend to more threads

Conclusions Offset causes variations in co-phase mix and therefore performance –Average 6.7% standard deviation in IPC A complete picture of performance can only be gained through looking at more than “0” offset Relative importance of co-phases can be determined mathematically