1. Pipelining review

2. Pipelining review Time IM Reg DM IM Reg DM IM Reg DM IM Reg DM
Clock Cycle 1
Clock Cycle 2
Clock Cycle 3
Clock Cycle 4
Clock Cycle 5
Clock Cycle 6
Clock Cycle 7
Time
instr1 IM
Reg DM
instr2 IM
Reg DM IM
Reg DM
instr3 IM
Reg DM
instr4
Instructions

3. Pipeline Control – big picture
Control Unit
ID/EX
EX/Mem
Mem/WB
IF/ID IF ID EX
Mem WB

4. Pipeline in action IF ID EX MEM WB Ctrl Instruction memory Register
file
clk
ALU
CLK SE PC
Data
Memory
0x1234
Let’s execute a program
at address 0x1234:
lw $1, 4($3)
add $2, $3, $4
or $3, $1, $2
bne $1, $3, L
Suppose: $1 = 1, $2 = 2, $3 = 1000, $4 = 6

5. Pipeline in action 1 IF ID EX MEM WB lw $1, 4($3) Instruction memory
Register
file
ALU
CLK SE PC
Data
Memory
0x1234
Clock: 1
$1 = 1, $2 = 2, $3 = 1000, $4 = 4

6. Pipeline in action 2 IF ID EX MEM WB add $2, $3, $4 Instruction memory
Register
file
ALU
CLK SE PC
Data
Memory
0x1238
lw $1, 4($3)
Clock: 1 2
$1 = 1, $2 =2, $3 =1000, $4 = 6

7. Pipeline in action 3 IF ID EX MEM WB or $3, $1, $2 Instruction memory
Register
file
ALU
CLK SE PC
Data
Memory
0x123C
add $2, $3, $4
lw $1, 4($3)
Clock: 1 2 3
$1 = 1, $2 =2, $3 =1000, $4 = 6

8. Pipeline in action 4 (anything wrong?)IF ID EX
MEM WB
bne $1, $3, L
Instruction
memory
Register
file
ALU
CLK SE PC
Data
Memory
0x1240
or $3, $1, $2
add $2, $3, $4
lw $1, 4($3)
Clock:
Let’s assume memory returns 42 for lw. 1 2 3 4
$1 = 1, $2 =2, $3 =1000, $4 = 6

9. Data Hazards! IF ID EX MEM WB bne $1, $3, L Instruction memory
Register
file
ALU
CLK SE PC
Data
Memory
0x1240
or $3, $1, $2
add $2, $3, $4
lw $1, 4($3)
Clock: 1 2 3 4
$1 = 1, $2 = 2, $3 = 1000, $4 = 6

10. Pipeline in action 5 IF ID EX MEM WB sll $4, $3, 5 Instruction memory
Register
file
ALU
CLK SE PC
Data
Memory
0x1244
bne $1, $3, L
or $3, $1, $2
add $2, $3, $4
Clock:
$1 and $2 both
wrong for or
because we got old values!
lw $1, 4($3) 1 2 3 4 5
$1 = 42, $2 = 2, $3 = 1000, $4 = 6

11. Data Hazards Data dependence Data dependence (RAW) occurs when:
result of an operation needed before it is stored back in reg. file:
lw $1, 4($3)
add $2, $3, $4
or $3, $1, $2
The data hazard above is read after write (RAW)
Data dependence (RAW) occurs when:
An instruction wants to read a register in stage 2, and
One instruction in stage 3 or stage 4 is going to write that register
Note: if the instruction writing the register is in stage 5, this is fine since we can write a register and read it in the same cycle
Hazard detection unit compares register fields across stages.

12. Resolving data dependencies
Have the compiler generate no-ops
Don’t have to deal with data hazards in hardware.
Stall the pipeline, i.e., create bubbles
the resulting delays are the same as for no-ops
Send the result generated in stage 3 or stage 4 to the appropriate input of the ALU.
This is forwarding or bypassing.
More performance at the cost of more hardware
For one simple pipeline, cost is slightly more control and extra buses
For several pipelines, say n, communication grows as O(n2)

13. Forwarding from WB to EXIF ID EX
MEM WB
sll $4, $3, 5
Instruction
memory
Register
file
ALU
CLK SE PC
Data
Memory
0x1244
bne $1, $3, L
or $3, $1, $2
add $2, $3, $4
Clock:
$1 fixed,
$2 still wrong
lw $1, 4($3) 1 2 3 4 5
$1 = 42, $2 = 2, $3 = 1000, $4 = 6

14. Forwarding from MEM to EXIF ID EX
MEM WB
sll $4, $3, 5
Instruction
memory
Register
file
ALU
CLK SE PC
Data
Memory
0x1244
bne $1, $3, L
or $3, $1, $2
add $2, $3, $4
Clock:
lw $1, 4($3) 1 2 3 4 5
$1 = 42, $2 = 2, $3 = 1000, $4 = 6

15. Generally, need to forward to both inputs!IF ID EX
MEM WB
sll $4, $3, 5
Instruction
memory
Register
file
ALU
CLK SE PC
Data
Memory
0x1244
or $3, $1, $2
beq $1, $3, L
add $2, $3, $4
What if we had: or $3, $1, $1
Clock:
lw $1, 4($3)
Need to forward $1 to both ALU inputs 1 2 3 4 5
$1 = 42, $2 = 2, $3 = 1000, $4 = 6

16. Forwarding a register twice?
What if had:
add $1, $2, $2
or $1, $3, $4
and $5, $1, $1
Where do you forward $1 from?

17. Another look at forwarding…
Clock Cycle 1
Clock Cycle 2
Clock Cycle 3
Clock Cycle 4
Clock Cycle 5
Clock Cycle 6
Clock Cycle 7
Time
lw $1, 4($3) IM
Reg DM
Forward $1
add $2, $3, $4 IM
Reg DM
Forward $2 IM
Reg DM
or $3, $1, $2
fwd $3 IM
Reg DM
bne $1, $3, L
Instructions

18. Data hazards and loads Time
Clock Cycle 1
Clock Cycle 2
Clock Cycle 3
Clock Cycle 4
Clock Cycle 5
Clock Cycle 6
Clock Cycle 7
Time
Can’t do forwarding in this case: can’t go back in time.
Have to insert a bubble between lw and or
lw $1, 4($3) IM
Reg DM ?
or $3, $1, $2 IM
Reg DM
What if we switch these? IM
Reg DM
add $2, $3, $4 IM
Reg DM
bne $1, $3, L
Instructions

19. Inserting a bubble
Have to do it when you load into a register which is used in the next instruction
Clock Cycle 1
Clock Cycle 2
Clock Cycle 3
Clock Cycle 4
Clock Cycle 5
Clock Cycle 6
Clock Cycle 7
Time
lw $1, 4($3) IM
Reg DM
Now we can forward $1
or $3, $1, $2 IM
Reg DM
do nothing IM
Reg DM
add $2, $3, $4
Insert bubble here IM
Reg DM
bne $1, $3, L
Instructions …

20. Other hazards We’ve seen data hazards Other types of hazards:
when an instruction in the pipeline is dependent on another instruction still in the pipeline.
Resolved by:
Bypassing/forwarding
Stalling
Other types of hazards:
Structural hazards
where two instructions at different stages want to use the same resource.
Solved by using more resources (e.g., instruction and data memory; several ALU’s). Won’t happen in our pipeline.
Control hazards
happens on a taken branch.
Evaluation of branch condition and calculation of branch target is not completed before next instruction is fetched.

21. Branch path IF ID EX MEM WB instr at L Instruction memory Register
file
ALU
CLK SE PC
sll $4, $3, 5
Data
Memory L:
This is wrong!
(Should be instr at L) =?
result of comparison
or $3, $1, $2
Determine new PC
add $2, $3, $4
Clock:
bne $1, $3, L 1 2 3 4 5 6
$1 = 42, $2 = 1006, $3 = 1000, $4 = 6

22. Resolving control hazards
Stall until result of the condition & target are known
too slow
Reduce penalty by redesigning the pipeline:
move branch calculation to ID stage
move branch comparison to ID stage
how to do quick compare?
use both edges of the clock
Delay slots
specify in ISA that instruction after branch is always executed.
Branch prediction
in IF stage, guess branch outcome
correct it if turns out to be wrong.

23. Lw & sw IF ID EX MEM WB Ctrl Instruction memory Register file ALU CLKSE PC
Data
Memory
Consider a memory copy program…
loop: lw $1, 0($3)
sw $1, 0($4)
addi $3, $3, 4
addi $4, $4, 4
bne $4, $5, Loop
Too many stalls!
How do we fix this?