1 COMP 206: Computer Architecture and Implementation Montek Singh Mon., Oct. 14, 2002 Topic: Instruction-Level Parallelism (Multiple-Issue, Speculation)

2 Hardware Support for More ILP  Speculation: allow an instruction to issue that is dependent on branch predicted to be taken without any consequences (including exceptions) if branch is not actually taken Hardware needs to provide an “undo” operation = squash Hardware needs to provide an “undo” operation = squash  Often try to combine with dynamic scheduling  Tomasulo: separate speculative bypassing of results from real bypassing of results When instruction no longer speculative, write results (instruction commit) When instruction no longer speculative, write results (instruction commit) execute out-of-order but commit in order execute out-of-order but commit in order Example: PowerPC 620, MIPS R10000, Intel P6, AMD K5 … Example: PowerPC 620, MIPS R10000, Intel P6, AMD K5 …

3 Hardware support for More ILP Need HW buffer for results of uncommitted instructions: reorder buffer (ROB) – –Reorder buffer can be operand source – –Once instruction commits, result is found in register – –3 fields: instr. type, destination, value – –Use reorder buffer number instead of reservation station – –Instructions commit in order – –As a result, its easy to undo speculated instructions on mispredicted branches or on exceptions Reorder Buffer FP Regs Instr Queue FP AdderFP Mult Res Stations Figure 3.29, page 228

4 Four Steps of Speculative Tomasulo Algorithm 1.Issue—get instruction from FP Op Queue If reservation station and reorder buffer slot free, issue instruction & send operands & reorder buffer no. for destination; each RS now also has a field for ROB#. If reservation station and reorder buffer slot free, issue instruction & send operands & reorder buffer no. for destination; each RS now also has a field for ROB#. 2.Execution—operate on operands (EX) When both operands ready then execute; if not ready, watch CDB for result; when both in reservation station, execute When both operands ready then execute; if not ready, watch CDB for result; when both in reservation station, execute 3.Write result—finish execution (WB) Write on Common Data Bus to all awaiting RS’s & ROB; mark RS available Write on Common Data Bus to all awaiting RS’s & ROB; mark RS available 4.Commit—update register with reorder result When instruction at head of reorder buffer & result present, update register with result (or store to memory) and remove instruction from reorder buffer When instruction at head of reorder buffer & result present, update register with result (or store to memory) and remove instruction from reorder buffer

5 Result Shift Register and Reorder Buffer  General solution to three problems Precise exceptions Precise exceptions Speculative execution Speculative execution Register renaming Register renaming  Solution in three steps In-order initiation, out-of-order termination (using RSRa) In-order initiation, out-of-order termination (using RSRa) In-order initiation, in-order termination (using RSRb) In-order initiation, in-order termination (using RSRb) In-order initiation, in-order termination, with renaming (using ROB) In-order initiation, in-order termination, with renaming (using ROB)  Architectural model Essentially MIPS FP pipeline Essentially MIPS FP pipeline Add takes 2 clock cycles, multiplication 5, division 10 Add takes 2 clock cycles, multiplication 5, division 10 Memory accesses take 1 clock cycle Memory accesses take 1 clock cycle Integer instructions take 1 clock cycle Integer instructions take 1 clock cycle 1 branch delay slot, delayed branches 1 branch delay slot, delayed branches

6 I-O Initiation, O-O Termination (RSRa) LOOP: LDF6, 32(R2) LDF2, 48(R3) MULTDF0, F2, F4 ADDIR2, R2, 8 ADDIR3, R3, 8 SUBDF8, F6, F2 DIVDF10, F10, F0 ADDDF6, F8, F6 BLEZR4, LOOP ADDIR4, R4, 1 LOOP: LDF6, 32(R2) LDF2, 48(R3) MULTDF0, F2, F4 ADDIR2, R2, 8 ADDIR3, R3, 8 SUBDF8, F6, F2 DIVDF10, F10, F0 ADDDF6, F8, F6 BLEZR4, LOOP ADDIR4, R4, 1

7 I-O Initiation, I-O Termination (RSRb)

8 Idea Behind ROB  Combine benefits of early issue and in-order update of state  Obtained from RSRa by adding a renaming mechanism to it  Add a FIFO to RSRa (implement as circular buffer)  When RSRa allows issuing of new instruction Enter instruction at tail of circular buffer Enter instruction at tail of circular buffer Buffer entry has multiple fields Buffer entry has multiple fields  [Result; Valid Bit; Destination Register Name; PC value; Exceptions]  Termination happens when result is produced, broadcast on CDB, written into circular buffer (replace M with T) Written ROB entry can serve as source of operands from now on Written ROB entry can serve as source of operands from now on  Commit happens when value is moved from circular buffer to register (replace W with C) Happens when instruction reaches head of circular buffer and has completed execution with no exceptions Happens when instruction reaches head of circular buffer and has completed execution with no exceptions

9 I-O Initiation, I-O Termination (ROB) LOOP: LDF6, 32(R2) LDF2, 48(R3) MULTDF0, F2, F4 ADDIR2, R2, 8 ADDIR3, R3, 8 SUBDF8, F6, F2 DIVDF10, F10, F0 ADDDF6, F8, F6 BLEZR4, LOOP ADDIR4, R4, 1 LOOP: LDF6, 32(R2) LDF2, 48(R3) MULTDF0, F2, F4 ADDIR2, R2, 8 ADDIR3, R3, 8 SUBDF8, F6, F2 DIVDF10, F10, F0 ADDDF6, F8, F6 BLEZR4, LOOP ADDIR4, R4, 1

10 States of Circular Buffer LOOP: LDF6, 32(R2) LDF2, 48(R3) MULTDF0, F2, F4 ADDIR2, R2, 8 ADDIR3, R3, 8 SUBDF8, F6, F2 DIVDF10, F10, F0 ADDDF6, F8, F6 BLEZR4, LOOP ADDIR4, R4, 1 LOOP: LDF6, 32(R2) LDF2, 48(R3) MULTDF0, F2, F4 ADDIR2, R2, 8 ADDIR3, R3, 8 SUBDF8, F6, F2 DIVDF10, F10, F0 ADDDF6, F8, F6 BLEZR4, LOOP ADDIR4, R4, 1 Entry in yellow is at head of buffer Entry in green is tail of buffer, i.e., next instruction goes here Greyed instructions have committed

11 Complexity of ROB  Assume dual-issue superscalar Load/Store machine with three-operand instructions Load/Store machine with three-operand instructions 64 registers 64 registers 16-entry circular buffer 16-entry circular buffer  Hardware support needed for ROB For each buffer entry For each buffer entry  One write port  Four read ports (two source operands of two instructions)  Four 6-bit comparators for associative lookup For each read port For each read port  16-way “priority” encoder with wrap-around (to get latest value)  Limited capacity of ROB is a structural hazard  Repeated writes to same register actually happen This is not the case in “classical” Tomasulo This is not the case in “classical” Tomasulo

12 Example: System Interactions Memory access time is 200 ns Which design is fastest?