1. CS252 Graduate Computer Architecture Lecture 8 ILP 2: Precise Interrupts and Getting the CPI < 1
John Kubiatowicz
Electrical Engineering and Computer Sciences
University of California, Berkeley

2. Review: Hardware techniques for out-of-order execution
HW exploitation of ILP
Works when can’t know dependence at compile time.
Code for one machine runs well on another
Scoreboard (ala CDC 6600 in 1963)
Centralized control structure
No register renaming, no forwarding
Pipeline stalls for WAR and WAW hazards.
Are these fundamental limitations??? (No)
Reservation stations (ala IBM 360/91 in 1966)
Distributed control structures
Implicit renaming of registers (dispatched pointers)
WAR and WAW hazards eliminated by register renaming
Results broadcast to all reservation stations for RAW
2/14/2007
CS252-s07, lecture 8

3. Review: Tomasulo Organization
FP Registers
From Mem
FP Op
Queue
Load Buffers
Load1
Load2
Load3
Load4
Load5
Load6
Store
Buffers
Add1
Add2
Add3
Mult1
Mult2
Resolve RAW memory conflict? (address in memory buffers)
Integer unit executes in parallel
Reservation
Stations
To Mem
FP adders
FP multipliers
Common Data Bus (CDB)
2/14/2007
CS252-s07, lecture 8

4. Review: Three Stages of Tomasulo Algorithm
1. Issue—get instruction from FP Op Queue
If reservation station free (no structural hazard), control issues instr & sends operands (renames registers).
2. Execution—operate on operands (EX)
When both operands ready then execute; if not ready, watch Common Data Bus for result
3. Write result—finish execution (WB)
Write on Common Data Bus to all awaiting units; mark reservation station available
Normal data bus: data + destination (“go to” bus)
Common data bus: data + source (“come from” bus)
64 bits of data + 4 bits of Functional Unit source address
Write if matches expected Functional Unit (produces result)
Does the broadcast
2/14/2007
CS252-s07, lecture 8

5. Review: Loop Example Cycle 9
Dataflow graph constructed completely in hardware
Renaming detaches early iterations from registers
2/14/2007
CS252-s07, lecture 8

6. Stream of Instructions
Problem: “Fetch” unit
Instruction Fetch
with
Branch Prediction
Out-Of-Order
Execution
Unit
Correctness Feedback
On Branch Results
Stream of Instructions
To Execute
Instruction fetch decoupled from execution
Often issue logic (+ rename) included with Fetch
2/14/2007
CS252-s07, lecture 8

7. Branches must be resolved quickly for loop overlap!
In our loop-unrolling example, we relied on the fact that branches were under control of “fast” integer unit in order to get overlap! Loop: LD F0 0 R1 MULTD F4 F0 F2 SD F4 0 R1 SUBI R1 R1 #8 BNEZ R1 Loop
What happens if branch depends on result of multd??
We completely lose all of our advantages!
Need to be able to “predict” branch outcome.
If we were to predict that branch was taken, this would be right most of the time.
Problem much worse for superscalar machines!
2/14/2007
CS252-s07, lecture 8

8. Prediction: Branches, Dependencies, Data
Prediction has become essential to getting good performance from scalar instruction streams.
We will discuss predicting branches. However, architects are now predicting everything: data dependencies, actual data, and results of groups of instructions:
At what point does computation become a probabilistic operation + verification?
We are pretty close with control hazards already…
Why does prediction work?
Underlying algorithm has regularities.
Data that is being operated on has regularities.
Instruction sequence has redundancies that are artifacts of way that humans/compilers think about problems.
Prediction  Compressible information streams?
2/14/2007
CS252-s07, lecture 8

9. What about Precise Exceptions/Interrupts?
Both Scoreboard and Tomasulo have:
In-order issue, out-of-order execution, out-of-order completion
Recall: An interrupt or exception is precise if there is a single instruction for which:
All instructions before that have committed their state
No following instructions (including the interrupting instruction) have modified any state.
Need way to resynchronize execution with instruction stream (I.e. with issue-order)
Easiest way is with in-order completion (i.e. reorder buffer)
Other Techniques (Smith paper): Future File, History Buffer
2/14/2007
CS252-s07, lecture 8

10. Reorder Buffer Idea: On issue: Done execute WB head of ROB
record instruction issue order
Allow them to execute out of order
Reorder them so that they commit in-order
On issue:
Reserve slot at tail of ROB
Record dest reg, PC
Tag u-op with ROB slot
Done execute
Deposit result in ROB slot
Mark exception state
WB head of ROB
Check exception, handle
Write register value, or
Commit the store
IFetch RF
Opfetch/Dcd
Write Back
2/14/2007
CS252-s07, lecture 8

11. Reorder Buffer + Forwarding
Idea:
Forward uncommitted results to later uncommitted operations
Trap
Discard remainder of ROB
Opfetch / Exec
Match source reg against all dest regs in ROB
Forward last (once available)
IFetch
Reg
Opfetch/Dcd
Write Back
2/14/2007
CS252-s07, lecture 8

12. Reorder Buffer + Forwarding + Speculation
Idea:
Issue branch into ROB
Mark with prediction
Fetch and issue predicted instructions speculatively
Branch must resolve before leaving ROB
Resolve correct
Commit following instr
Resolve incorrect
Mark following instr in ROB as invalid
Let them clear
IFetch
Reg
Opfetch/Dcd
Write Back
2/14/2007
CS252-s07, lecture 8

13. History File Maintain issue order, like ROB
Each entry records dest reg and old value of dest. Register
What if old value not available when instruction issues?
FUs write results into register file
Forward into correct entry in history file
When exception reaches head
Restore architected registers from tail to head
IFetch
Opfetch/Dcd
Reg
Write Back
2/14/2007
CS252-s07, lecture 8

14. Future file Idea Arch registers reflect state at commit point
Future register reflect whatever instructions have completed
On WB update future
On commit update arch
On exception
Discard future
Replace with arch
Dest w/I ROB
IFetch
Reg
Opfetch/Dcd
Future
Write Back
2/14/2007
CS252-s07, lecture 8

15. What are the hardware complexities with reorder buffer (ROB)?FP Op
Queue
FP Adder
Res Stations
FP Regs
Compar network
Reorder Table
Dest Reg
Result
Exceptions?
Valid
Program Counter
How do you find the latest version of a register?
As specified by Smith paper, need associative comparison network
Could use future file or just use the register result status buffer to track which specific reorder buffer has received the value
Need as many ports on ROB as register file
2/14/2007
CS252-s07, lecture 8

16. Recall: Four Steps of Speculative Tomasulo Algorithm
1. Issue—get instruction from FP Op Queue
If reservation station and reorder buffer slot free, issue instr & send operands & reorder buffer no. for destination (this stage sometimes called “dispatch”)
2. Execution—operate on operands (EX)
When both operands ready then execute; if not ready, watch CDB for result; when both in reservation station, execute; checks RAW (sometimes called “issue”)
3. Write result—finish execution (WB)
Write on Common Data Bus to all awaiting FUs & reorder buffer; mark reservation station available.
4. Commit—update register with reorder result
When instr. at head of reorder buffer & result present, update register with result (or store to memory) and remove instr from reorder buffer. Mispredicted branch flushes reorder buffer (sometimes called “graduation”)
2/14/2007
CS252-s07, lecture 8

17. Tomasulo With Reorder buffer:
Done?
FP Op
Queue
ROB7
ROB6
ROB5
ROB4
ROB3
ROB2
ROB1
Newest
Reorder Buffer
Oldest F0
LD F0,10(R2) N
Registers To
Memory
Dest
Resolve RAW memory conflict? (address in memory buffers)
Integer unit executes in parallel
Dest
from
Memory
Dest
Reservation
Stations
1 10+R2
FP adders
FP multipliers
2/14/2007
CS252-s07, lecture 8

18. Tomasulo With Reorder buffer:
Done?
FP Op
Queue
F10 F0
ADDD F10,F4,F0
LD F0,10(R2) N
ROB7
ROB6
ROB5
ROB4
ROB3
ROB2
ROB1
Newest
Reorder Buffer
Oldest
Registers To
Memory
Dest
Resolve RAW memory conflict? (address in memory buffers)
Integer unit executes in parallel
Dest
from
Memory
2 ADDD R(F4),ROB1
Dest
Reservation
Stations
1 10+R2
FP adders
FP multipliers
2/14/2007
CS252-s07, lecture 8

19. Tomasulo With Reorder buffer:
Done?
FP Op
Queue F2
F10 F0
DIVD F2,F10,F6
ADDD F10,F4,F0
LD F0,10(R2) N
ROB7
ROB6
ROB5
ROB4
ROB3
ROB2
ROB1
Newest
Reorder Buffer
Oldest
Registers To
Memory
Dest
Resolve RAW memory conflict? (address in memory buffers)
Integer unit executes in parallel
Dest
from
Memory
2 ADDD R(F4),ROB1
3 DIVD ROB2,R(F6)
Dest
Reservation
Stations
1 10+R2
FP adders
FP multipliers
2/14/2007
CS252-s07, lecture 8

20. Tomasulo With Reorder buffer:
Done?
FP Op
Queue F0
ADDD F0,F4,F6 N F4
LD F4,0(R3) --
BNE F2,<…> F2
F10
DIVD F2,F10,F6
ADDD F10,F4,F0
LD F0,10(R2)
ROB7
ROB6
ROB5
ROB4
ROB3
ROB2
ROB1
Newest
Reorder Buffer
Oldest
Registers To
Memory
Dest
Resolve RAW memory conflict? (address in memory buffers)
Integer unit executes in parallel
Dest
from
Memory
2 ADDD R(F4),ROB1
6 ADDD ROB5, R(F6)
3 DIVD ROB2,R(F6)
Dest
Reservation
Stations
1 10+R2
6 0+R3
FP adders
FP multipliers
2/14/2007
CS252-s07, lecture 8

21. Tomasulo With Reorder buffer:
Done?
FP Op
Queue -- F0
ROB5
ST 0(R3),F4
ADDD F0,F4,F6 N F4
LD F4,0(R3)
BNE F2,<…> F2
F10
DIVD F2,F10,F6
ADDD F10,F4,F0
LD F0,10(R2)
ROB7
ROB6
ROB5
ROB4
ROB3
ROB2
ROB1
Newest
Reorder Buffer
Oldest
Registers To
Memory
Dest
Resolve RAW memory conflict? (address in memory buffers)
Integer unit executes in parallel
Dest
from
Memory
2 ADDD R(F4),ROB1
6 ADDD ROB5, R(F6)
3 DIVD ROB2,R(F6)
Dest
Reservation
Stations
1 10+R2
6 0+R3
FP adders
FP multipliers
2/14/2007
CS252-s07, lecture 8

22. Tomasulo With Reorder buffer:
Done?
FP Op
Queue -- F0
M[10]
ST 0(R3),F4
ADDD F0,F4,F6 Y N F4
LD F4,0(R3)
BNE F2,<…> F2
F10
DIVD F2,F10,F6
ADDD F10,F4,F0
LD F0,10(R2)
ROB7
ROB6
ROB5
ROB4
ROB3
ROB2
ROB1
Newest
Reorder Buffer
Oldest
Registers To
Memory
Dest
Resolve RAW memory conflict? (address in memory buffers)
Integer unit executes in parallel
Dest
from
Memory
2 ADDD R(F4),ROB1
6 ADDD M[10],R(F6)
3 DIVD ROB2,R(F6)
Dest
Reservation
Stations
1 10+R2
FP adders
FP multipliers
2/14/2007
CS252-s07, lecture 8

23. Tomasulo With Reorder buffer:
Done?
FP Op
Queue -- F0
M[10]
<val2>
ST 0(R3),F4
ADDD F0,F4,F6 Y Ex F4
LD F4,0(R3)
BNE F2,<…> N F2
F10
DIVD F2,F10,F6
ADDD F10,F4,F0
LD F0,10(R2)
ROB7
ROB6
ROB5
ROB4
ROB3
ROB2
ROB1
Newest
Reorder Buffer
Oldest
Registers To
Memory
Dest
Resolve RAW memory conflict? (address in memory buffers)
Integer unit executes in parallel
Dest
from
Memory
2 ADDD R(F4),ROB1
3 DIVD ROB2,R(F6)
Dest
Reservation
Stations
1 10+R2
FP adders
FP multipliers
2/14/2007
CS252-s07, lecture 8

24. Tomasulo With Reorder buffer:
Done?
FP Op
Queue -- F0
M[10]
<val2>
ST 0(R3),F4
ADDD F0,F4,F6 Y Ex F4
LD F4,0(R3)
BNE F2,<…> N
ROB7
ROB6
ROB5
ROB4
ROB3
ROB2
ROB1
Newest
What about memory
hazards???
Reorder Buffer F2
DIVD F2,F10,F6 N
F10
ADDD F10,F4,F0 N
Oldest F0
LD F0,10(R2) N
Registers To
Memory
Dest
Resolve RAW memory conflict? (address in memory buffers)
Integer unit executes in parallel
Dest
from
Memory
2 ADDD R(F4),ROB1
3 DIVD ROB2,R(F6)
Dest
Reservation
Stations
1 10+R2
FP adders
FP multipliers
2/14/2007
CS252-s07, lecture 8

25. Memory Disambiguation: Sorting out RAW Hazards in memory
Question: Given a load that follows a store in program order, are the two related?
(Alternatively: is there a RAW hazard between the store and the load)? Eg: st 0(R2),R ld R6,0(R3)
Can we go ahead and start the load early?
Store address could be delayed for a long time by some calculation that leads to R2 (divide?).
We might want to issue/begin execution of both operations in same cycle.
Today: Answer is that we are not allowed to start load until we know that address 0(R2)  0(R3)
Next Week: We might guess at whether or not they are dependent (called “dependence speculation”) and use reorder buffer to fixup if we are wrong.
2/14/2007
CS252-s07, lecture 8

26. Hardware Support for Memory Disambiguation
Need buffer to keep track of all outstanding stores to memory, in program order.
Keep track of address (when becomes available) and value (when becomes available)
FIFO ordering: will retire stores from this buffer in program order
When issuing a load, record current head of store queue (know which stores are ahead of you).
When have address for load, check store queue:
If any store prior to load is waiting for its address, stall load.
If load address matches earlier store address (associative lookup), then we have a memory-induced RAW hazard:
store value available  return value
store value not available  return ROB number of source
Otherwise, send out request to memory
Actual stores commit in order, so no worry about WAR/WAW hazards through memory.
2/14/2007
CS252-s07, lecture 8

27. Memory Disambiguation:
Done?
FP Op
Queue
ROB7
ROB6
ROB5
ROB4
ROB3
ROB2
ROB1
Newest
Reorder Buffer --
LD F4, 10(R3) N F2
R[F5]
ST 10(R3), F5 N F0
LD F0,32(R2) N
Oldest --
<val 1>
ST 0(R3), F4 Y
Registers To
Memory
Dest
Resolve RAW memory conflict? (address in memory buffers)
Integer unit executes in parallel
Dest
from
Memory
Dest
Reservation
Stations
2 32+R2
4 ROB3
FP adders
FP multipliers
2/14/2007
CS252-s07, lecture 8

28. Relationship between precise interrupts and speculation:
Speculation is a form of guessing
Branch prediction, data prediction
If we speculate and are wrong, need to back up and restart execution to point at which we predicted incorrectly
This is exactly same as precise exceptions!
Branch prediction is a very important!
Need to “take our best shot” at predicting branch direction.
If we issue multiple instructions per cycle, lose lots of potential instructions otherwise:
Consider 4 instructions per cycle
If take single cycle to decide on branch, waste from instruction slots!
Technique for both precise interrupts/exceptions and speculation: in-order completion or commit
This is why reorder buffers in all new processors
2/14/2007
CS252-s07, lecture 8

29. Administrative
Midterm I: Wednesday 3/14 Location: 306 Soda Hall TIME: 5:30 - 8:30
Can have 1 sheet of 8½x11 handwritten notes – both sides
No microfiche of the book!
This info is on the Lecture page (has been)
Meet at LaVal’s afterwards for Pizza and Beverages
Great way for me to get to know you better
I’ll Buy!
2/14/2007
CS252-s07, lecture 8

30. Quick Recap: Explicit Register Renaming
Make use of a physical register file that is larger than number of registers specified by ISA
Keep a translation table:
ISA register => physical register mapping
When register is written, replace table entry with new register from freelist.
Physical register becomes free when not being used by any instructions in progress.
Fetch
Decode/
Rename
Execute
Rename
Table
2/14/2007
CS252-s07, lecture 8

31. Explicit register renaming: R10000 Freelist Management
Current Map Table
Done?
Oldest
Newest
P32
P34
P36
P38 
P60
P62
Freelist
Resolve RAW memory conflict? (address in memory buffers)
Integer unit executes in parallel
Physical register file larger than ISA register file
On issue, each instruction that modifies a register is allocated new physical register from freelist
Used on: R10000, Alpha 21264, HP PA8000
2/14/2007
CS252-s07, lecture 8

32. Explicit register renaming: R10000 Freelist Management
Current Map Table
Done?
Oldest
Newest
P34
P36
P38
P40 
P60
P62
Freelist F0 P0
LD P32,10(R2) N
Resolve RAW memory conflict? (address in memory buffers)
Integer unit executes in parallel
Note that physical register P0 is “dead” (or not “live”) past the point of this load.
When we go to commit the load, we free up
2/14/2007
CS252-s07, lecture 8

33. Explicit register renaming: R10000 Freelist Management
Current Map Table
Done?
Oldest
Newest
P36
P38
P40
P42 
P60
P62
F10
P10
ADDD P34,P4,P32 N
Freelist F0 P0
LD P32,10(R2) N
Resolve RAW memory conflict? (address in memory buffers)
Integer unit executes in parallel
2/14/2007
CS252-s07, lecture 8

34. Explicit register renaming: R10000 Freelist Management
Current Map Table -- F2
F10 F0 P2
P10 P0
BNE P36,<…> N
DIVD P36,P34,P6
ADDD P34,P4,P32
LD P32,10(R2)
Done?
Oldest
Newest
P38
P40
P44
P48 
P60
P62
Freelist
Resolve RAW memory conflict? (address in memory buffers)
Integer unit executes in parallel
P32
P36 P4 F6 F8
P34
P12
P14
P16
P18
P20
P22
P24
p26
P28
P30 
P38
P40
P44
P48
P60
P62
Checkpoint at BNE instruction
2/14/2007
CS252-s07, lecture 8

35. Explicit register renaming: R10000 Freelist Management
Current Map Table
Done? --
ST 0(R3),P40 Y
Oldest
Newest F0
P32
ADDD P40,P38,P6 Y F4 P4
LD P38,0(R3) Y
P42
P44
P48
P50  P0
P10 --
BNE P36,<…> N F2 P2
DIVD P36,P34,P6 N
F10
P10
ADDD P34,P4,P32 y
Freelist F0 P0
LD P32,10(R2) y
Resolve RAW memory conflict? (address in memory buffers)
Integer unit executes in parallel
P32
P36 P4 F6 F8
P34
P12
P14
P16
P18
P20
P22
P24
p26
P28
P30 
P38
P40
P44
P48
P60
P62
Checkpoint at BNE instruction
2/14/2007
CS252-s07, lecture 8

36. Explicit register renaming: R10000 Freelist Management
Current Map Table
Done?
Oldest
Newest
P38
P40
P44
P48  P0
P10 F2 P2
DIVD P36,P34,P6 N
F10
P10
ADDD P34,P4,P32 y
Freelist F0 P0
LD P32,10(R2) y
Error fixed by restoring map table and merging freelist
Resolve RAW memory conflict? (address in memory buffers)
Integer unit executes in parallel
P32
P36 P4 F6 F8
P34
P12
P14
P16
P18
P20
P22
P24
p26
P28
P30 
P38
P40
P44
P48
P60
P62
Checkpoint at BNE instruction
2/14/2007
CS252-s07, lecture 8

37. Advantages of Explicit Renaming
Decouples renaming from scheduling:
Pipeline can be exactly like “standard” DLX pipeline (perhaps with multiple operations issued per cycle)
Or, pipeline could be tomasulo-like or a scoreboard, etc.
Standard forwarding or bypassing could be used
Allows data to be fetched from single register file
No need to bypass values from reorder buffer
This can be important for balancing pipeline
Many processors use a variant of this technique:
R10000, Alpha 21264, HP PA8000
Another way to get precise interrupt points:
All that needs to be “undone” for precise break point is to undo the table mappings
Provides an interesting mix between reorder buffer and future file
Results are written immediately back to register file
Registers names are “freed” in program order (by ROB)
2/14/2007
CS252-s07, lecture 8

38. Getting CPI < 1: Issuing Multiple Instructions/Cycle
Two variations
Superscalar: varying no. instructions/cycle (1 to 8), scheduled by compiler or by HW (Tomasulo)
IBM PowerPC, Sun UltraSparc, DEC Alpha, HP 8000
(Very) Long Instruction Words (V)LIW: fixed number of instructions (4-16) scheduled by the compiler; put ops into wide templates
Joint HP/Intel agreement in 1999/2000?
Intel Architecture-64 (IA-64) 64-bit address
Style: “Explicitly Parallel Instruction Computer (EPIC)”
Anticipated success lead to use of Instructions Per Clock cycle (IPC) vs. CPI
2/14/2007
CS252-s07, lecture 8

39. Getting CPI < 1: Issuing Multiple Instructions/Cycle
Superscalar DLX: 2 instructions, 1 FP & 1 anything else
– Fetch 64-bits/clock cycle; Int on left, FP on right
– Can only issue 2nd instruction if 1st instruction issues
– More ports for FP registers to do FP load & FP op in a pair
Type Pipe Stages
Int. instruction IF ID EX MEM WB
FP instruction IF ID EX MEM WB
Int. instruction IF ID EX MEM WB
FP instruction IF ID EX MEM WB
Int. instruction IF ID EX MEM WB
FP instruction IF ID EX MEM WB
1 cycle load delay expands to 3 instructions in SS
instruction in right half can’t use it, nor instructions in next slot
2/14/2007
CS252-s07, lecture 8

40. Review: Unrolled Loop that Minimizes Stalls for Scalar
1 Loop: LD F0,0(R1)
2 LD F6,-8(R1)
3 LD F10,-16(R1)
4 LD F14,-24(R1)
5 ADDD F4,F0,F2
6 ADDD F8,F6,F2
7 ADDD F12,F10,F2
8 ADDD F16,F14,F2
9 SD 0(R1),F4
10 SD -8(R1),F8
11 SD -16(R1),F12
12 SUBI R1,R1,#32
13 BNEZ R1,LOOP
14 SD 8(R1),F16 ; 8-32 = -24
14 clock cycles, or 3.5 per iteration
LD to ADDD: 1 Cycle
ADDD to SD: 2 Cycles
2/14/2007
CS252-s07, lecture 8

41. Loop Unrolling in Superscalar
Integer instruction FP instruction Clock cycle
Loop: LD F0,0(R1) 1
LD F6,-8(R1) 2
LD F10,-16(R1) ADDD F4,F0,F2 3
LD F14,-24(R1) ADDD F8,F6,F2 4
LD F18,-32(R1) ADDD F12,F10,F2 5
SD 0(R1),F4 ADDD F16,F14,F2 6
SD -8(R1),F8 ADDD F20,F18,F2 7
SD (R1),F12 8
SD (R1),F16 9
SUBI R1,R1,#40 10
BNEZ R1,LOOP 11
SD (R1),F20 12
Unrolled 5 times to avoid delays (+1 due to SS)
12 clocks, or 2.4 clocks per iteration (1.5X)
2/14/2007
CS252-s07, lecture 8

42. Dynamic Scheduling in Superscalar
How to issue two instructions and keep in-order instruction issue for Tomasulo?
Assume 1 integer + 1 floating point
1 Tomasulo control for integer, 1 for floating point
Issue 2X Clock Rate, so that issue remains in order
Only FP loads might cause dependency between integer and FP issue:
Replace load reservation station with a load queue; operands must be read in the order they are fetched
Load checks addresses in Store Queue to avoid RAW violation
Store checks addresses in Load Queue to avoid WAR,WAW
Called “decoupled architecture”
2/14/2007
CS252-s07, lecture 8

43. Multiple Issue Challenges
While Integer/FP split is simple for the HW, get CPI of 0.5 only for programs with:
Exactly 50% FP operations
No hazards
If more instructions issue at same time, greater difficulty of decode and issue:
Even 2-scalar => examine 2 opcodes, 6 register specifiers, & decide if 1 or 2 instructions can issue
Multiported rename logic: must be able to rename same register multiple times in one cycle!
Rename logic one of key complexities in the way of multiple issue!
VLIW: tradeoff instruction space for simple decoding
The long instruction word has room for many operations
By definition, all the operations the compiler puts in the long instruction word are independent => execute in parallel
E.g., 2 integer operations, 2 FP ops, 2 Memory refs, 1 branch
16 to 24 bits per field => 7*16 or 112 bits to 7*24 or 168 bits wide
Need compiling technique that schedules across several branches
2/14/2007
CS252-s07, lecture 8

44. Loop Unrolling in VLIW Unrolled 7 times to avoid delays
Memory Memory FP FP Int. op/ Clock reference 1 reference 2 operation 1 op. 2 branch
LD F0,0(R1) LD F6,-8(R1) 1
LD F10,-16(R1) LD F14,-24(R1) 2
LD F18,-32(R1) LD F22,-40(R1) ADDD F4,F0,F2 ADDD F8,F6,F2 3
LD F26,-48(R1) ADDD F12,F10,F2 ADDD F16,F14,F2 4
ADDD F20,F18,F2 ADDD F24,F22,F2 5
SD 0(R1),F4 SD -8(R1),F8 ADDD F28,F26,F2 6
SD -16(R1),F12 SD -24(R1),F
SD -32(R1),F20 SD -40(R1),F24 SUBI R1,R1,#48 8
SD -0(R1),F28 BNEZ R1,LOOP 9
Unrolled 7 times to avoid delays
7 results in 9 clocks, or 1.3 clocks per iteration (1.8X)
Average: 2.5 ops per clock, 50% efficiency
Note: Need more registers in VLIW (15 vs. 6 in SS)
2/14/2007
CS252-s07, lecture 8

45. Recall: Software Pipelining with Loop Unrolling in VLIW
Memory Memory FP FP Int. op/ Clock
reference 1 reference 2 operation 1 op. 2 branch
LD F0,-48(R1) ST 0(R1),F4 ADDD F4,F0,F2 1
LD F6,-56(R1) ST -8(R1),F8 ADDD F8,F6,F2 SUBI R1,R1,#24 2
LD F10,-40(R1) ST 8(R1),F12 ADDD F12,F10,F2 BNEZ R1,LOOP 3
Software pipelined across 9 iterations of original loop
In each iteration of above loop, we:
Store to m,m-8,m-16 (iterations I-3,I-2,I-1)
Compute for m-24,m-32,m-40 (iterations I,I+1,I+2)
Load from m-48,m-56,m-64 (iterations I+3,I+4,I+5)
9 results in 9 cycles, or 1 clock per iteration
Average: 3.3 ops per clock, 66% efficiency
Note: Need less registers for software pipelining
(only using 7 registers here, was using 15)
2/14/2007
CS252-s07, lecture 8

46. Advantages of HW (Tomasulo) vs. SW (VLIW) Speculation
HW determines address conflicts
HW better branch prediction
HW maintains precise exception model
HW does not execute bookkeeping instructions
Works across multiple implementations
SW speculation is much easier for HW design
2/14/2007
CS252-s07, lecture 8

47. Superscalar v. VLIW Smaller code size
Binary compatability across generations of hardware
Simplified Hardware for decoding, issuing instructions
No Interlock Hardware (compiler checks?)
More registers, but simplified Hardware for Register Ports (multiple independent register files?)
2/14/2007
CS252-s07, lecture 8

48. Intel/HP “Explicitly Parallel Instruction Computer (EPIC)”
3 Instructions in 128 bit “groups”; field determines if instructions dependent or independent
Smaller code size than old VLIW, larger than x86/RISC
Groups can be linked to show independence > 3 instr
64 integer registers + 64 floating point registers
Not separate filesper funcitonal unit as in old VLIW
Hardware checks dependencies (interlocks => binary compatibility over time)
Predicated execution (select 1 out of 64 1-bit flags) => 40% fewer mispredictions?
IA-64 : instruction set architecture; EPIC is type
Merced is name of first implementation (1999/2000?)
LIW = EPIC?
2/14/2007
CS252-s07, lecture 8

49. Summary #1
Dynamic hardware schemes can unroll loops dynamically in hardware
Form of limited dataflow
Reorder Buffer
In-order issue, Out-of-order execution, In-order commit
Holds results until they can be commited in order
Serves as source of info until instructions committed
Provides support for precise exceptions/Speculation: simply throw out instructions later than excepted instruction.
Memory Disambiguation:
Tracking of RAW hazards through memory
Keep program-order queue of stores When have address for load, check store queue:
If any store prior to load is waiting for its address, stall load.
If load address matches earlier store address (associative lookup), then we have a memory-induced RAW hazard:
Otherwise, send out request to memory
2/14/2007
CS252-s07, lecture 8

50. Summary #2
Explicit Renaming: more physical registers than needed by ISA.
Separates renaming from scheduling
Opens up lots of options for resolving RAW hazards
Rename table: tracks current association between architectural registers and physical registers
Potentially complicated rename table management
Superscalar and VLIW: CPI < 1 (IPC > 1)
Dynamic issue vs. Static issue
More instructions issue at same time => larger hazard penalty
Limitation is often number of instructions that you can successfully fetch and decode per cycle  “Flynn barrier”
Other models of parallelism: Vector processing
2/14/2007
CS252-s07, lecture 8