Handling Branches in TLS Systems with Multi-Path Execution Polychronis Xekalakis and Marcelo Cintra University of Edinburgh

HPCA Introduction  Power efficiency, complexity and time-to-market reasons lead to CMPs  Many simple cores = high TLP but low ILP –Ok for throughput computing and embarrassingly parallel applications  Problem: –No benefits for sequential applications –Even for mostly parallel applications Amdahl’s Law limits performance gains with many cores  Solution: Speculative Multithreading (SM)

Speculative Multithreading  Basic Idea: Use idle cores/contexts to speculate on future application needs –TLS: speculatively execute parallel threads –HT/RA: speculatively perform future memory operations –MP: speculatively execute along multiple branch targets  No SM model works best all times  Hardware infrastructure is very similar  Our Idea: Combine SM models and seamlessly exploit (speculative) TLP and/or ILP –In this work: TLS + MP –(for TLS +HT/RA see [ICS’09]) ICS 20093

4 Key Contributions  Analyze branch prediction for TLS Systems  Propose a mixed execution model that combines TLS with MP execution  We show that TLS allows MP to be more aggressive  Our approach outperforms state-of-the-art SM models: –TLS by 9.2% avg. (up to 23.2%) –MP by 28.2 % avg. (up to 138%) HPCA 2010

5 Outline  Introduction  Speculative Multithreaded Models  Analysis of Branch Prediction in TLS  Mixed Execution Model  Experimental Setup and Results  Conclusions HPCA 2010

6 Thread Level Speculation  Compiler deals with: –Task selection –Code generation  HW deals with: –Different context –Spawn threads –Detecting violations –Replaying –Arbitrate commit HPCA 2010 Thread 1 Thread 2 Speculative Time

7 Thread Level Speculation  Benefit: TLP/ILP –TLP (Overlapped Execution) –ILP (Prefetching) HPCA 2010 Thread 1 Thread 2 Speculative Time Overlapped Execution Thread 1 Thread 2 Speculative Time Prefetching

MultiPath Execution  Compiler deals with: –Nothing  HW deals with: –Different context –When to do MP –Discard wrong path 8HPCA 2010 Main Thread MP Mode Time Correct Paths Wrong Paths

MultiPath Execution  Benefit: –ILP (Branch Pred.) 9HPCA 2010 Main Thread Time Correct Paths Wrong Paths Branch Misp. Cost

10 Outline  Introduction  Speculative Multithreaded Models  Analysis of Branch Prediction in TLS  Mixed Execution Model  Experimental Setup and Results  Conclusions HPCA 2010

Impact of Branch Prediction on TLS  TLS emulates wider processor: –Removing mispredictions important (Amdahl) 11HPCA 2010

Branch Entropy for TLS  Much harder for TLS: –History partitioning –History re-order 12HPCA 2010

Increasing the Size of the Branch Predictor  Aliasing not much of a problem  Fundamental limitation is lack of history 13HPCA 2010

Designing a Better Predictor  Predictors that exploit longer histories not necessarily better.. 14HPCA 2010

15 Outline  Introduction  Speculative Multithreaded Models  Analysis of Branch Prediction in TLS  Mixed Execution Model  Experimental Setup and Results  Conclusions HPCA 2010

Mixed Execution Model  When idle resources: – Try MP on top of TLS!!  Map TLS threads on empty cores  Map MP threads on empty contexts (same core)  Minimal extra HW: –Branch confidence estimator –MP bit – thread on MP mode –PATHS – how many outstanding branches –DIR – which path thread followed 16HPCA 2010

Combined TLS/MP Model 17HPCA 2010 Thread 1 Thread 2 Speculative Time

Combined TLS/MP Model 18HPCA 2010 Thread 1 Thread 2 Speculative Time Low Confidence Branch Thread 1 MP: 0 PATHS: 000 DIR: 000

Combined TLS/MP Model 19HPCA 2010 Thread 1a Thread 2 Speculative Time Multi-Path Mode Thread 1a MP: 1 PATHS: 001 DIR: 000 Thread 1b MP: 1 PATHS: 001 DIR: 001 Thread 1b

Combined TLS/MP Model 20HPCA 2010 Thread 1a Thread 2 Speculative Time Branch Resolved Thread 1b Thread 1a MP: 1 PATHS: 001 DIR: 000 Thread 1b MP: 0 PATHS: 000 DIR: 000

Intricacies to be Handled  How do we map TLS/MP threads? –Different mapping policies for TLS threads  Dealing with thread ordering –Correct data forwarding  Dealing with violations –While in “MP-Mode” delay restarts/kills/commits –No squashes on the wrong path  Thread spawning: –Delayed as well – keep contention low 21HPCA 2010

22 Outline  Introduction  Speculative Multithreaded Models  Analysis of Branch Prediction in TLS  Mixed Execution Model  Experimental Setup and Results  Conclusions HPCA 2010

23 Experimental Setup  Simulator, Compiler and Benchmarks: –SESC ( –POSH (Liu et al. PPoPP ‘06) –Spec 2000 Int.  Architecture: –Four way CMP, 4-Issue cores, 6 contexts / core –32K-bit OGEHL, 1KByte BTB, 32-Entry RAS –8 Kbit enhanced JRS confidence estimator –32KB L1 Data (multi-versioned) and Instruction Caches –1MB unified L2 Caches HPCA 2010

24 Comparing TLS, MP and Combined TLS/MP HPCA 2010

25 Comparing TLS, MP and Combined TLS/MP  Additive benefits; no point in doubling the predictor HPCA 2010

26 Comparing TLS, MP and Combined TLS/MP  Additive benefits; no point in doubling the predictor  9.2% over TLS, 28.2% over MP HPCA 2010

Pipeline Flushes  Significant amount of flush reductions  More than base MP! 27HPCA 2010

28 Outline  Introduction  Speculative Multithreaded Models  Analysis of Branch Prediction in TLS  Mixed Execution Model  Experimental Setup and Results  Conclusions HPCA 2010

Also in the Paper …  Detailed HW description  Impact of scheduling  Limiting MP to DP  Effect of scaling  Effect of a better CE 29HPCA 2010

30 Conclusions  CMPs are here to stay: – What about single threaded apps. and apps with significant seq. sections? –We advocate the use of speculative multithreading  Analyzed branch prediction for modern TLS systems  Proposed a new mixed execution model –TLS is nicely complemented by MP  Unified scheme outperforms existing SM models –TLS by 9.2% avg. (up to 23.2%) –MP by 28.2 % avg. (up to 138%) HPCA 2010

Handling Branches in TLS Systems with Multi-Path Execution Polychronis Xekalakis and Marcelo Cintra University of Edinburgh

Backup Slides ICS

Prediction Stats ICS Stat. (%)Bzip2CraftyGapGzipMcfParserTwolfVortexVprAvg. Misp PVN PVP SPEC SENS

Performance Model  Sall = Sseq x Silp x Sovl 1.Compute overall speedup (Sall) 34 Tseq/Tmt

Performance Model  Sall = Sseq x Silp x Sovl 1.Compute overall speedup (Sall) 2.Compute sequential TLS speedup (Sseq) 35 Tseq/T1p

Performance Model  Sall = Sseq x Silp x Sovl 1.Compute overall speedup (Sall) 2.Compute sequential TLS speedup (Sseq) 3.Compute speedup due to ILP (Silp) 36 (T1+T2)/(T1’+T2’)

Performance Model  Sall = Sseq x Silp x Sovl 1.Compute overall speedup (Sall) 2.Compute sequential TLS speedup (Sseq) 3.Compute speedup due to ILP (Silp) 4.Use everything to compute TLP (Sovl) 37 Sall/(Sseq x Silp)