1. CDA 3101 Summer 2003 Introduction to Computer Organization Pipeline Control And Pipeline Hazards 17 July 2003

2. Control Signals PC Add Instr. Mem Data Mem Regs Sign extend ALU MuxMux 4 IF/ID ID/EX EX/MEM MEM/WB Add Shift left 2 ALU Control MuxMux MuxMux MuxMux Branch Zero MemWrite MemRead MemtoReg PCSrc ALUSrc ALUOp RegDst rt[20-16] rd[15-11] RegWrite

3. ALU Control Input InstructionALUOpFunction codeALU ActionALU Control Lw00xxxxxxAdd010 Sw00xxxxxxAdd010 Beq01xxxxxxSubtract110 Add10100000Add010 Sub10100010Subtract110 And10100100And000 Or10100101Or001 Slt10101010Set on less than111

4. Control Lines Instruction Execution stage control lines Memory access control lines WriteBack control lines Reg Dst ALU Op1 ALU Op2 ALU Src BranchMem Read Mem Write Reg Write Mem2 Reg R-format110000010 Lw000101011 Swx0010010x Beqx0101000x

5. Control Implementation Pipelining leaves the meaning of the 9 control lines unchanged Set control lines (to defined values) in each stage for each instruction Extend pipeline registers to include control information Nothing to control during IF and ID Create control information during ID

6. Generation/Propagation of Control IF/IDID/EXEX/MEMMEM/WB Control WB M EX Instruction WB M

7. PC Add Instr. Mem Data Mem Regs Sign extend ALU MuxMux 4 IF/ID ID/EX EX/MEM MEM/WB Add Shift left 2 ALU Control MuxMux MuxMux MuxMux Branch Zero MemWrite MemRead MemtoReg PCSrc ALUSrc ALUOp RegDst rt[20-16] rd[15-11] RegWrite Control WBWB M E WBWB M WBWB

8. Example lw$10, 20($1) sub$11, $2, $3 and$12, $4, $5 or$13, $6, $7 add$14, $8, $9

9. Cycle 1

10. Cycle 2

11. Cycle 3

12. Cycle 4

13. Cycle 5

14. Cycle 6

15. Limits to Pipelining Hazards prevent next instruction from executing during its designated clock cycle –Structural hazards HW cannot support this combination of instructions Ex: Single person to fold and put clothes away –Control hazards Branches stall the pipeline until the hazard “bubbles” in the pipeline –Data hazards Instruction depends on result of prior instruction Ex: Missing sock

16. Pipeline Hazards (Example) Jim’s green socks : one in other in depends on  stall since folder busy TaskOrderTaskOrder B C D A E F bubble 12 2 AM 6 PM 7 8 9 10 11 1 Time 30 A A D D Bag A: Control puts 90m bubble in pipeline be- tween dryer and folder (done 9pm) Bag D: Cannot complete until 10:30pm (one folder available)

17. Structural Hazard 1: Single Memory IM = DM => Read same memory twice in one clock cycle I$ Load Instr 1 Instr 2 Instr 3 Instr 4 ALU I$ Reg D$Reg ALU I$ Reg D$Reg ALU I$ Reg D$Reg ALU Reg D$Reg ALU I$ Reg D$Reg I n s t r. O r d e r Time (clock cycles)

18. Structural Hazard 2: Register File Try read and write to registers simultaneously I$ Load Instr 1 Instr 2 Instr 3 Instr 4 ALU I$ Reg D$Reg ALU I$ Reg D$Reg ALU I$ Reg D$Reg ALU Reg D$Reg ALU I$ Reg D$Reg I n s t r. O r d e r Time (clock cycles)

19. Structural Hazards: Solutions Structural hazard 1: single memory –Two memories? infeasible and inefficient => Two Level 1 caches (instruction and data) Structural hazard 2: register file –Register access takes less that ½ ALU stage time => Use the following convention: Always Write during first half of each cycle Always Read during second half of each cycle –Both, Read and Write can be performed during the same clock cycle (a small delay between)

20. Control Hazard: Branch Instr. (1/2) Branch decision-making hardware in ALU stage –Two more instructions after the branch will always be fetched, whether or not the branch is taken Desired functionality of a branch –if we do not take the branch, don’t waste any time and continue executing normally –if we take the branch, don’t execute any instructions after the branch, just go to the desired label

21. Control Hazard: Branch Instr. (2/2) Initial Solution: Stall until decision is made –Insert “no-op” instructions: those that accomplish nothing, just take time –Drawback: branches take 3 clock cycles each (assuming comparator is put in ALU stage) Better Solution: Move comparator to Stage 2 –Benefit: since branch is complete in Stage 2, only one unnecessary instruction is fetched –Therefore, only one no-op is needed –This means that branches are idle in Stages 3, 4 and 5.

22. Control Hazard: Better Sol’n. Move comparator up to Stage 2 Benefit: since branch is complete in Stage 2, only one unnecessary instruction is fetched, so only one no-op is needed This means that branches are idle in Stages 3, 4 and 5. Add Beq Load ALU I$ Reg D$Reg ALU I$ Reg D$Reg ALU Reg D$Reg I$ I n s t r. O r d e r bub ble Time (clock cycles)

23. Best: Delayed Branches (1/2) If we take the branch, none of the instructions after the branch get executed by accident New definition: whether or not we take the branch, the instruction immediately following the branch gets executed (called the branch-delay slot)

24. Best: Delayed Branches (2/2) Notes on Branch-Delay Slot –Worst-Case Scenario: can always use a no-op –Better Case: can find an instruction preceding the branch which can be placed in the branch- delay slot without affecting flow of the program Re-ordering instructions is a common speedup technique – done in compiler Compiler must be smart in order to find instructions to do this Usually can find such an instruction at least 50% of the time - REAL STUFF!!

25. Nondelayed vs. Delayed Nondelayed BranchDelayed Branch add $1,$2,$3 sub $4, $5,$6 beq $1, $4, Exit or $8, $9,$10 xor $10, $1,$11 Exit:... add $1,$2,$3 sub $4, $5,$6 beq $1, $4, Exit or $8, $9,$10 xor $10, $1,$11 Exit: ...

26. Conclusions (1/2) Optimal Pipeline –Each stage is executing part of an instruction each cycle. –One instruction finishes during each clock cycle. –On average, execute far more quickly What makes this work? –Similarities between instructions –Each stage takes about the same amount of time as all others

27. Conclusions (2/2) Pipelining a Big Idea: widely used concept What makes it less than perfect? –Structural hazards:  Need more HW resources –Control hazards:  D elayed branch –Data hazards: an instruction depends on a previous one Next Time: Pipeline Performance Issues