1. ICS’02 1 Low-Complexity Reorder Buffer Architecture* *supported in part by DARPA through the PAC-C program and NSF Gurhan Kucuk, Dmitry Ponomarev, Kanad Ghose Department of Computer Science State University of New York Binghamton, NY 13902-6000 http://www.cs.binghamton.edu/~lowpower 16 th Annual ACM International Conference on Supercomputing (ICS’02), June 24 th 2002

2. ICS’02 2 Outline ROB complexities Motivation for the low-complexity ROB Low-complexity ROB design Results Concluding remarks

3. ICS’02 3 What This Work is All About Complex, richly-ported ROBs are common in modern superscalar datapaths Number of ports are aggravated when results are held within ROB slots (Example: Pentium III) ROB complexity reduction is important for reducing power and improving performance ROB dissipates a non-trivial fraction of the total chip power ROB accesses stretch over several cycles Goal of this work: Reduce the complexity and power dissipation of the ROB without sacrificing performance

4. ICS’02 4 Pentium III-like Superscalar Datapath IQ Function Units Instruction Issue F1D1 FU1 FU2 FUm ARF Result/status forwarding buses EX Instruction dispatch Architectural Register File F2 Fetch Decode/Dispatch D2 D-cache LSQ ROB

5. ICS’02 5 ROB Port Requirements for a W-way CPU ROB Writeback W write ports to write results Dispatch/Issue 2W read ports to read the source operands Decode/Dispatch W write ports to setup entries Commit W read ports for instruction commitment

6. ICS’02 6 ROB Port Requirements for a W-way CPU ROB Writeback W write ports To write results Dispatch/Issue 2W read ports to read the source operands Decode/Dispatch 1 W-wide write port to setup entries Commit 1 W-wide read port for instruction commitment

7. ICS’02 7 Where are the Source Values Coming From? IQ Function Units Instruction Issue F1D1 FU1 FU2 FUm ARF Result/status forwarding buses EX Instruction dispatch Architectural Register File F2 Fetch Decode/Dispatch D2 D-cache LSQ ROB 1 2 3

8. ICS’02 8 Where are the Source Values Coming From ? 96-entry ROB, 4-way processor SPEC2K Benchmarks 62%32%6%

9. ICS’02 9 How Efficiently are the Ports Used ? ROB Writeback W write ports To write results Dispatch/Issue 2W read ports to read the source operands Decode/Dispatch W write ports to setup entries Commit W read ports for instruction commitment 6%

10. ICS’02 10 Approaches to Reducing ROB Complexity Reduce the number of read ports for reading out the source operand values More radical (and better): Completely eliminate the read ports for reading source operand values!

11. ICS’02 11 Reducing the Number of Read Ports Performance Drop % 3.5%1.0% Average IPC Drop: bzip2gapgccgzipmcfparserperltwolfInt Avg.vortexvpr appluapsiartequakemesamgridswimwupwiseFP Avg.

12. ICS’02 12 Problems with Retaining Fewer Source Read Ports on the ROB Need arbitration for the small number of ports Additional logic needed to block the instructions which could not get the port. Need a switching network to route the operands to correct destinations Multi-cycle access still remains in the critical path of Dispatch/Issue logic

13. ICS’02 13 Our Solution: Elimination of Read Ports IQ Function Units Instruction Issue F1D1 FU1 FU2 FUm ARF Result/status forwarding buses EX Instruction dispatch Architectural Register File F2 Fetch Decode/Dispatch D2 D-cache LSQ ROB 1 2 3

14. ICS’02 14 Our Solution: Elimination of Read Ports IQ Function Units Instruction Issue F1D1 FU1 FU2 FUm ARF Result/status forwarding buses EX Instruction dispatch Architectural Register File F2 Fetch Decode/Dispatch D2 D-cache LSQ ROB 1 2 3

15. ICS’02 15 Our Solution: Elimination of Read Ports IQ Function Units Instruction Issue F1D1 FU1 FU2 FUm ARF Result/status forwarding buses EX Instruction dispatch Architectural Register File F2 Fetch Decode/Dispatch D2 D-cache LSQ 1 3 ROB

16. ICS’02 16 Comparison of ROB Bitcells (0.18µ, TSMC) Layout of a 32-ported SRAM bitcell Layout of a 16-ported SRAM bitcell Area Reduction – 71% Shorter bit and wordlines

17. ICS’02 17 Our Solution: Elimination of Read Ports IQ Function Units Instruction Issue F1D1 FU1 FU2 FUm ARF Result/status forwarding buses EX Instruction dispatch Architectural Register File F2 Fetch Decode/Dispatch D2 D-cache LSQ ROB Area Reduction – 45%

18. ICS’02 18 Eliminating/Reducing the Number of Read Ports: Effects on Power Dissipation Power is reduced because: shorter bitlines and wordlines lower capacitive loading fewer decoders fewer drivers and sense amps

19. ICS’02 19 Completely Eliminating the Source Read Ports on the ROB The Problem: Issue of instructions that require a value stored in the ROB will stall Solutions: Forward the value to the waiting instruction at the time of committing the value: LATE FORWARDING

20. ICS’02 20 Late Forwarding: Use the Normal Forwarding Buses! IQ Function Units Instruction Issue F1D1 FU1 FU2 FUm ARF EX Instruction dispatch Architectural Register File F2 Fetch Decode/Dispatch D2 D-cache LSQ ROB Result/status forwarding buses:

21. ICS’02 21 IQ Function Units Instruction Issue F1D1 FU1 FU2 FUm ARF EX Instruction dispatch Architectural Register File F2 Fetch Decode/Dispatch D2 D-cache LSQ ROB Result/status forwarding buses: Late Forwarding: Use the Normal Forwarding Buses!

22. ICS’02 22 Optimizing Late Forwarding PROBLEM: If Late Forwarding is done for every result that is committed, additional forwarding buses are needed in order not to degrade the performance SOLUTION: Selective Late Forwarding (SLF) SLF requires additional bit in the ROB That bit is set by the dispatched instructions that require Late Forwarding No additional forwarding buses are needed, since SLF traffic is very small

23. ICS’02 23 IQ Function Units Instruction Issue F1D1 FU1 FU2 FUm ARF Only 3.5% of the traffic is from SELECTIVE LATE FORWARDING EX Instruction dispatch Architectural Register File F2 Fetch Decode/Dispatch D2 D-cache LSQ ROB Result/status forwarding buses: Late Forwarding: Use the Normal Forwarding Buses!

24. ICS’02 24 Performance Drop of Simplified ROB Performance Drop % 9.6%3.5%1.0% Average IPC Drop: bzip2gapgccgzipmcfparserperltwolfInt Avg.vortexvpr appluapsiartequakemesamgridswimwupwiseFP Avg. 37% 17%

25. ICS’02 25 IPC Penalty: Source Value Not Accessible within the ROB Forwarding Late Forwarding/ Commitment Lifetime of a Result Value Result Generation time Value within ARF Value within ROB

26. ICS’02 26 Improving IPC with No Read Ports Cache recently generated values in a set of RETENTION LATCHES (RL) Retention Latches are SMALL and FAST Only 8 to 16 latches needed in the set Entire set has 1 or 2 read ports

27. ICS’02 27 Datapath with the Retention Latches IQ Function Units Instruction Issue F1D1 FU1 FU2 FUm ARF Result/status forwarding buses EX Instruction dispatch F2 Fetch Decode/Dispatch D2 D-cache LSQ ROB Architectural Register File

28. ICS’02 28 Datapath with the Retention Latches IQ Function Units Instruction Issue F1D1 FU1 FU2 FUm ARF Result/status forwarding buses EX Instruction dispatch Architectural Register File F2 Fetch Decode/Dispatch D2 D-cache LSQ RETENTION LATCHES ROB

29. ICS’02 29 The Structure of the Retention Latch Set L ROB slot addresses (L=1 or 2) L-ported CAM field (key = ROB_slot_id) W write ports for writing up to W results in parallel Status L recently-written results (L=1 or 2 works great) Result Values 8 or 16 latches

30. ICS’02 30 Retention Latch Management Strategies FIFO 8 entry RL: 42% hit rate 16 entry RL: 55% hit rate LRU 8 entry RL: 56% hit rate 16 entry RL: 62% hit rate Random Replacement Worse performance than FIFO

31. ICS’02 31 Hit Ratios to Retention Latches 42%55%56%62% Hit Ratios bzip2gapgccgzipmcfparserperltwolfInt Avg.vortexvpr appluapsiartequakemesamgridswimwupwiseFP Avg. Average Hit Ratio:

32. ICS’02 32 Accessing Retention Latch Entries ROB index is used as a unique key in the Retention Latches to search the result values Need to maintain unique keys even when we have: Reuse of a ROB slot: Not a problem for FIFO simply flush a RL entry at commit time for LRU Branch mispredictions

33. ICS’02 33 Handling Branch Mispredictions Selective RL Flushing: Retention latch entries that are in the mispredicted path are flushed Uses branch tags Complicated implementation Complete RL Flushing: All retention latch entries are flushed Very simple implementation Performance drop is only 1.5% compared to selective flushing

34. ICS’02 34 Misprediction Handling: Performance 1.5% Average IPC Drop: IPC

35. ICS’02 35 Scenario 1: Traditional Design 5 ROB index Src1 valid ? Src1 value ? ? ? Src2 valid Src2 value Simplified IDB entry #1 Src2 arch. 3 Src1 arch. 2 ADD Instruction Instruction: ADD R1, R2, R3

36. ICS’02 36 Scenario 1: Traditional Design 5 ROB index Src1 valid ? Src1 value ? ? ? Src2 valid Src2 value Simplified IDB entry #1 Src2 reg. 3 Src1 reg. 2 ADD Instruction Instruction: ADD R1, R2, R3 Arch. ROB=0 ARF=1 0 1 2 3 4 … …… …… …… … … 12 3 0 1 ROB# /Phys. Rename Table

37. ICS’02 37 Scenario 1: Traditional Design 5 ROB index Src1 valid ? Src1 value ? ? Src2 valid Src2 value Simplified IDB entry #1 Src2 reg. 3 Src1 reg. 2 ADD Instruction Instruction: ADD R1, R2, R3 Arch. ROB# /Phys. ROB=0 ARF=1 0 1 2 3 4 … …… …… …… … … 12 3 0 1 ? ROB# /Phys. Phys. valid Phys. value ……… 12 ……… 17 Rename Table ROB

38. ICS’02 38 Scenario 1: Traditional Design 5 ROB index Src1 valid 1 Src1 value 7 ? Src2 valid Src2 value Simplified IDB entry #1 Src2 reg. 3 Src1 reg. 2 ADD Instruction Instruction: ADD R1, R2, R3 Arch. ROB# /Phys. ROB=0 ARF=1 0 1 2 3 4 … …… …… …… … … 12 3 0 1 ? ROB# /Phys. Phys. valid Phys. value ……… 12 ……… 17 Rename Table ROB

39. ICS’02 39 Scenario 1: Traditional Design 5 ROB index Src1 valid ? Src1 value ? ? Src2 valid Src2 value Simplified IDB entry #1 Src2 reg. 3 Src1 reg. 2 ADD Instruction Instruction: ADD R1, R2, R3 Arch. ROB# /Phys. ROB=0 ARF=1 0 1 2 3 4 … …… …… …… … … 12 3 0 1 ? ROB# /Phys. Phys. valid Phys. value ……… 12 ……… 0? Rename Table ROB

40. ICS’02 40 Scenario 1: Traditional Design 5 ROB index Src1 valid 0 Src1 value ? ? Src2 valid Src2 value Simplified IDB entry #1 Src2 reg. 3 Src1 reg. 2 ADD Instruction Instruction: ADD R1, R2, R3 Arch. ROB# /Phys. ROB=0 ARF=1 0 1 2 3 4 … …… …… …… … … 12 3 0 1 ? ROB# /Phys. Phys. valid Phys. value ……… 12 ……… 0? Rename Table ROB

41. ICS’02 41 Scenario 1: Traditional Design 5 ROB index Src1 valid 1 Src1 value 7 ? Src2 valid Src2 value Simplified IDB entry #1 Src2 reg. 3 Src1 reg. 2 ADD Instruction Instruction: ADD R1, R2, R3 Arch. ROB# /Phys. ROB=0 ARF=1 0 1 2 3 4 … …… …… …… … … 12 3 0 1 ? Arch. value …… 3 …… 43 Rename Table ARF

42. ICS’02 42 Scenario 1: Traditional Design 5 ROB index Src1 valid 1 Src1 value 7 43 Src2 valid Src2 value Simplified IDB entry #1 Src2 reg. 3 Src1 reg. 2 ADD Instruction Instruction: ADD R1, R2, R3 Arch. ROB# /Phys. ROB=0 ARF=1 0 1 2 3 4 … …… …… …… … … 12 3 0 1 1 Arch. value …… 3 …… 43 Rename Table ARF

43. ICS’02 43 Scenario 2: Simplified ROB with RLs 5 ROB index Src1 valid ? Src1 value ? ? ? Src2 valid Src2 value Simplified IDB entry #1 Src2 arch. 3 Src1 arch. 2 ADD Instruction Instruction: ADD R1, R2, R3

44. ICS’02 44 Scenario 2: Simplified ROB with RLs 5 ROB index Src1 valid ? Src1 value ? ? ? Src2 valid Src2 value Simplified IDB entry #1 Src2 reg. 3 Src1 reg. 2 ADD Instruction Instruction: ADD R1, R2, R3 Arch. ROB=0 ARF=1 0 1 2 3 4 … …… …… …… … … 12 3 0 1 ROB# /Phys. Rename Table

45. ICS’02 45 Scenario 2: Simplified ROB with RLs 5 ROB index Src1 valid ? Src1 value ? ? Src2 valid Src2 value Simplified IDB entry #1 Src2 reg. 3 Src1 reg. 2 ADD Instruction Instruction: ADD R1, R2, R3 Arch. ROB# /Phys. ROB=0 ARF=1 0 1 2 3 4 … …… …… …… … … 12 3 0 1 ? ROB# /Phys. Phys. value …… 12 …… 7 Rename Table Retention Latches

46. ICS’02 46 Scenario 2: Simplified ROB with RLs 5 ROB index Src1 valid 1 Src1 value 7 ? Src2 valid Src2 value Simplified IDB entry #1 Src2 reg. 3 Src1 reg. 2 ADD Instruction Instruction: ADD R1, R2, R3 Arch. ROB# /Phys. ROB=0 ARF=1 0 1 2 3 4 … …… …… …… … … 12 3 0 1 ? Rename Table ROB# /Phys. Phys. value …… 12 …… 7 Retention Latches

47. ICS’02 47 Scenario 2: Simplified ROB with RLs 5 ROB index Src1 valid ? Src1 value ? ? Src2 valid Src2 value Simplified IDB entry #1 Src2 reg. 3 Src1 reg. 2 ADD Instruction Instruction: ADD R1, R2, R3 Arch. ROB# /Phys. ROB=0 ARF=1 0 1 2 3 4 … …… …… …… … … 12 3 0 1 ? Rename Table ROB# /Phys. Phys. value …… … …… … MISS Retention Latches

48. ICS’02 48 Scenario 2: Simplified ROB with RLs 5 ROB index Src1 valid 0 Src1 value ? ? Src2 valid Src2 value Simplified IDB entry #1 Src2 reg. 3 Src1 reg. 2 ADD Instruction Instruction: ADD R1, R2, R3 Arch. ROB# /Phys. ROB=0 ARF=1 0 1 2 3 4 … …… …… …… … … 12 3 0 1 ? ROB# /Phys. Phys. valid Phys. value ……… 12 ……… XX Rename Table ROB ROB# /Phys. Phys. value …… … …… … Retention Latches MISS X: Don’t Care SLF … … 0

49. ICS’02 49 Scenario 2: Simplified ROB with RLs 5 ROB index Src1 valid 0 Src1 value ? ? Src2 valid Src2 value Simplified IDB entry #1 Src2 reg. 3 Src1 reg. 2 ADD Instruction Instruction: ADD R1, R2, R3 Arch. ROB# /Phys. ROB=0 ARF=1 0 1 2 3 4 … …… …… …… … … 12 3 0 1 ? ROB# /Phys. Phys. valid Phys. value ……… 12 ……… XX Rename Table ROB ROB# /Phys. Phys. value …… … …… … Retention Latches MISS X: Don’t Care SLF … … 1

50. ICS’02 50 Scenario 2: Simplified ROB with RLs 5 ROB index Src1 valid 1 Src1 value 7 ? Src2 valid Src2 value Simplified IDB entry #1 Src2 reg. 3 Src1 reg. 2 ADD Instruction Instruction: ADD R1, R2, R3 Arch. ROB# /Phys. ROB=0 ARF=1 0 1 2 3 4 … …… …… …… … … 12 3 0 1 ? Arch. value …… 3 …… 43 Rename Table ARF

51. ICS’02 51 Scenario 2: Simplified ROB with RLs 5 ROB index Src1 valid 1 Src1 value 7 43 Src2 valid Src2 value Simplified IDB entry #1 Src2 reg. 3 Src1 reg. 2 ADD Instruction Instruction: ADD R1, R2, R3 Arch. ROB# /Phys. ROB=0 ARF=1 0 1 2 3 4 … …… …… …… … … 12 3 0 1 1 Arch. value …… 3 …… 43 Rename Table ARF

52. ICS’02 52 Experimental Setup: the AccuPower (DATE’02) Compiled SPEC benchmarks Datapath specs Performance stats VLSI layout data SPICE deck SPICE Microarchitectural Simulator (Rooted in SimpleScalar) Energy/Power Estimator Power/energy stats SPICE measures of energy per transition Transition counts, Context information

53. ICS’02 53 Configuration of the Simulated System Machine width4-way Issue Queue32 entries 96 entriesReorder Buffer Load/Store Queue 32 entries Simulated the execution of SPEC2000 benchmarks

54. ICS’02 54 Assumed Timings Rename Table lookup for ROB index Rename Table Lookup for ROB index Associative lookup of operand from retention latches using ROB index as a key Source operand read from the ROB Source operand read from the ROB Smaller delay: few latches D1 D2D3 D1 D2 Timing of the baseline modelTiming of the simplified ROB

55. ICS’02 55 Experimental Results: Effect on Performance Performance Drop % 0.1%-1.6%-1.0%-2.3% appluapsiartequakemesamgridswimwupwiseFP Avg. bzip2gapgccgzipmcfparserperltwolfInt Avg.vortexvpr Avg. IPC Drop:

56. ICS’02 56 Experimental Results: Effect on Performance Performance Drop % 3.3%1.7%2.3%1.0% appluapsiartequakemesamgridswimwupwiseFP Avg. bzip2gapgccgzipmcfparserperltwolfInt Avg.vortexvpr Avg. IPC Drop:

57. ICS’02 57 Experimental Results: Effect on Power Power Savings % bzip2gapgccgzipmcfparserperltwolfInt Avg.vortexvpr appluapsiartequakemesamgridswimwupwiseFP Avg. 30%23.4%22.2%21%20.2% Avg. Savings:

58. ICS’02 58 Summary of Results Significantly reduced ROB complexity and power dissipation 45% area reduction 20% to 30% power reduction across SPEC 2000 benchmarks Actual IPC improvements: 1.6% to 2.3% gain across SPEC benchmarks IPC gains come from 1 cycle access to RL (vs. 2 cycles that would be needed for ROB access)

59. ICS’02 59 Related Work Value-Aging Buffer (Hu & Martonosi, PACS 2000) Forwarding Buffer and Clustered Register Cache (Borch et.al., HPCA’02) Multiple Register Banks (Cruz et.al., ISCA’00 & Balasubramonian et.al., MICRO’01) See paper for discussions

60. ICS’02 60 Conclusions Typical source operand location statistics can be successfully exploited to reduce ROB complexity Significant reduction in ROB area and power – no ROB ports needed for reading source operands IPC gains are possible because of the use of a small sized, low-ported Retention Latch to supply cached operand values in a single cycle

61. ICS’02 61 Low-Complexity Reorder Buffer Architecture* *supported in part by DARPA through the PAC-C program and NSF Gurhan Kucuk, Dmitry Ponomarev, Kanad Ghose Department of Computer Science State University of New York Binghamton, NY 13902-6000 http://www.cs.binghamton.edu/~lowpower 16 th Annual ACM International Conference on Supercomputing (ICS’02), June 24 th 2002