1. Implicitly-Multithreaded Processors Il Park and Babak Falsafi and T. N. Vijaykumar Presented by: Ashay Rane Published in: SIGARCH Computer Architecture News, 2003

2. Agenda  Overview (IMT, state-of-art) ‏  IMT enhancements  Key results  Critique  Relation to Term Project

3. Implicitly Multithreaded Processor (IMT) ‏  SMT with speculation  Optimizations to basic SMT support  Average perf. improvement of 24% Max: 69%

4. State-of-the-art  Pentium 4 HT  IBM POWER5  MIPS MT

5. Speculative SMT operation  When branch encountered, start executing likely path “speculatively” i.e. allow for rollback (thread squash) in certain circumstances (misprediction, dependence)  Overcome cost, overhead with savings in execution time and power (but worth the effort) ‏  Complication because commit by independent threads (buffer for each thread). Also issue, register renaming, cache & TLB conflicts.  If dependence violation, squash thread and restart execution

6. How to buffer speculative data?  Load/Store Queue (LSQ) ‏  Buffers data (along with its address) ‏  Helps enforce dependency check  Makes rollback possible  Cache-based approaches

7. IMT : Most significant improvements  Assistance from Multiscalar compiler  Resource- and dependence-aware fetch policy  Multiplexing threads on a single hardware context  Overlapping thread startup operations with previous threads execution

8. What does Compiler do?  Extracts threads from program (loops) ‏  Generates thread descriptor data about registers read and written and control flow exits (for rename tables)  Annotates instructions with special codes (“forward” & “release”) for dependence checking

9. Fetch Policy  Hardware keeps track of resource utilization  Resource requirement prediction from past four execution instances  When dependencies exist (detected from compiler- generated data), bias towards non-speculative threads  Goal is to reduce number of thread squashes

10. Multiplexing threads on a single hardware context  Observations:  Threads usually short  Number of contexts less (2-8) ‏ Hence frequent switching, less overlap

11. Multiplexing (contd.) ‏  Larger threads can lead to:  Speculation buffer overflow  Increased dependence mis-speculation  Hence thread squashing  Each execution context can further support multiple threads (3-6) ‏

12. Multiplexing: Required Hardware  Per context per thread:  Program Counter  Register rename table  LSQ shared among threads running on 1 execution context

13. Multiplexing: Implementation Issues  LSQ shared but it needs to maintain loads and stores for each thread separately  Therefore, create “gaps” for yet-to-be-fetched instructions / data  If space falls short, squash subsequent thread  What if threads from one program are mapped to different contexts?  IMT searches through other contexts  Easier to have multiple LSQs per context per thread but not good cost and power consumption

14. Register renaming  Required because multiple threads may use same registers  Separate rename tables  Master Rename Table (global) ‏  Local Rename Table (per thread) ‏  Pre-assign table (per thread) ‏

15. Register renaming: Flow  Thread Invocation:  Copy from Master table into Local table (to reflect current status) ‏  Also use “create” and “use” mask of thread descriptor (to for dependence check) ‏  Before every subsequent thread invocation:  Pre-assign rename maps into Pre-assign table  Copy from Pre-assign table to Master table and mark registers as “busy”. So no successor thread can use them before current thread writes to them.

16. Hiding thread startup delay  Rename tables to be setup before execution begins  Occupies table bandwidth, hence cannot be done for a number of threads in parallel  Hence overlap setting up of rename tables with previous thread’s execution

17. Load/Store Queue  Per context  Speculative load / store: Search through current and other contexts for dependence  No searching for non-speculative loads  Searching can take time, so schedules load-dependent instructions accordingly

18. Key Results

19.  Average improvement: 24%  Reduction in data dependence stalls  Little overhead of optimizations  Not all benchmark programs

20. Assuming 2-3 threads per context, 6-8 LSQ entries per thread. Performance relative to IMT with unlimited resources

21. ICOUNT: Favor least number of instructions remaining to be executed Biased-ICOUNT: Favor non-speculative threads Worst-case resource estimation Reduced thread squashing

22. TME: Executes both paths of an unpredictable branch (but such branches uncommon)‏ DMT: –Hardware-selection of threads. So spawns threads on backward- branch or function call instead of loops. –Also spawns threads out of order. So lower accuracy of branch prediction.

23. Critique

24. Compiler Support  Improvement in applications compiled using Multiscalar compiler  Scientific computing applications, not for desktop applications

25. LSQ Limitations  LSQ size deciding the size of speculative thread  Pentium 4 (without SMT): 48 Loads, 24 Stores  Pentium 4 HT: 24 Loads, 12 Stores per thread  IBM Power5: 32 Loads, 32 Stores per thread

26. LSQ Limitations: Alternative  Cache-based approach i.e. Partition the cache to support different versions  Extra support required, but scalable

27. Register file size  IMT considers register file sizes of 128 and up.  Pentium 4 (as well as HT): Register file size = 128  IBM POWER5: Register file size = 80

28. Searching LSQ  Since loads and stores organized as per thread, search involves all locations of other threads.  If loads/stores organized according to addresses then lesser values to search.  Can make use of associativity of cache

29. Searching LSQ (contd.) ‏

30. So how is performance still high?  Assistance from Compiler  Resource and dependency-aware fetching  Multiple threads on an execution context  Overlapping rename table creation with execution

31. Term project  “Cache-based throughput improvement techniques for Speculative SMT processors”  Optimizations from IMT  Increasing granularity to reduce number of thread squashes

32. Thank you