1. CS 152 Computer Architecture and Engineering Lecture 4 - Pipelining Krste Asanovic Electrical Engineering and Computer Sciences University of California at Berkeley http://www.eecs.berkeley.edu/~krste http://inst.eecs.berkeley.edu/~cs152

2. 2/3/2009CS152-Spring’09 2 Last time in Lecture 3 Microcoding became less attractive as gap between RAM and ROM speeds reduced Complex instruction sets difficult to pipeline, so difficult to increase performance as gate count grew Iron-law explains architecture design space –Trade instructions/program, cycles/instruction, and time/cycle Load-Store RISC ISAs designed for efficient pipelined implementations –Very similar to vertical microcode –Inspired by earlier Cray machines

3. 2/3/2009CS152-Spring’09 3 “Iron Law” of Processor Performance Time = Instructions Cycles Time Program Program * Instruction * Cycle –Instructions per program depends on source code, compiler technology, and ISA –Cycles per instructions (CPI) depends upon the ISA and the microarchitecture –Time per cycle depends upon the microarchitecture and the base technology MicroarchitectureCPIcycle time Microcoded>1short Single-cycle unpipelined1long Pipelined1short This lecture

4. 2/3/2009CS152-Spring’09 4 An Ideal Pipeline All objects go through the same stages No sharing of resources between any two stages Propagation delay through all pipeline stages is equal The scheduling of an object entering the pipeline is not affected by the objects in other stages stage 1 stage 2 stage 3 stage 4 These conditions generally hold for industrial assembly lines. But can an instruction pipeline satisfy the last condition?

5. 2/3/2009CS152-Spring’09 5 Pipelined MIPS To pipeline MIPS: First build MIPS without pipelining with CPI=1 Next, add pipeline registers to reduce cycle time while maintaining CPI=1

6. 2/3/2009CS152-Spring’09 6 Pipelined Datapath Clock period can be reduced by dividing the execution of an instruction into multiple cycles t C > max {t IM, t RF, t ALU, t DM, t RW } ( = t DM probably) However, CPI will increase unless instructions are pipelined write -back phase fetch phase execute phase decode & Reg-fetch phase memory phase addr wdata rdata Data Memory we ALU Imm Ext 0x4 Add addr rdata Inst. Memory rd1 GPRs rs1 rs2 ws wd rd2 we IR PC

7. 2/3/2009CS152-Spring’09 7 Technology Assumptions Thus, the following timing assumption is reasonable A small amount of very fast memory (caches) backed up by a large, slower memory Fast ALU (at least for integers) Multiported Register files (slower!) t IM t RF t ALU t DM t RW A 5-stage pipelined Harvard architecture will be the focus of our detailed design

8. 2/3/2009CS152-Spring’09 8 5-Stage Pipelined Execution time t0t1t2t3t4t5t6t7.... instruction1IF 1 ID 1 EX 1 MA 1 WB 1 instruction2 IF 2 ID 2 EX 2 MA 2 WB 2 instruction3IF 3 ID 3 EX 3 MA 3 WB 3 instruction4 IF 4 ID 4 EX 4 MA 4 WB 4 instruction5 IF 5 ID 5 EX 5 MA 5 WB 5 Write - Back (WB) I-Fetch (IF) Execute (EX) Decode, Reg. Fetch (ID) Memory (MA) addr wdata rdata Data Memory we ALU Imm Ext 0x4 Add addr rdata Inst. Memory rd1 GPRs rs1 rs2 ws wd rd2 we IR PC

9. 2/3/2009CS152-Spring’09 9 5-Stage Pipelined Execution Resource Usage Diagram time t0t1t2t3t4t5t6t7.... IFI 1 I 2 I 3 I 4 I 5 IDI 1 I 2 I 3 I 4 I 5 EX I 1 I 2 I 3 I 4 I 5 MA I 1 I 2 I 3 I 4 I 5 WB I 1 I 2 I 3 I 4 I 5 Resources Write - Back (WB) I-Fetch (IF) Execute (EX) Decode, Reg. Fetch (ID) Memory (MA) addr wdata rdata Data Memory we ALU Imm Ext 0x4 Add addr rdata Inst. Memory rd1 GPRs rs1 rs2 ws wd rd2 we IR PC

10. 2/3/2009CS152-Spring’09 10 Pipelined Execution: ALU Instructions IR 31 PC A B Y R MD1 MD2 addr inst Inst Memory 0x4 Add IR Imm Ext ALU rd1 GPRs rs1 rs2 ws wd rd2 we wdata addr wdata rdata Data Memory we Not quite correct! We need an Instruction Reg (IR) for each stage

11. 2/3/2009CS152-Spring’09 11 Pipelined MIPS Datapath without jumps IR 31 PC A B Y R MD1 MD2 addr inst Inst Memory 0x4 Add IR Imm Ext ALU rd1 GPRs rs1 rs2 ws wd rd2 we Data Memory wdata addr wdata rdata we OpSel ExtSelBSrc WBSrc MemWrite RegDst RegWrite FDEMW Control Points Need to Be Connected

12. 2/3/2009CS152-Spring’09 12 How Instructions can Interact with each other in a pipeline An instruction in the pipeline may need a resource being used by another instruction in the pipeline  structural hazard An instruction may depend on something produced by an earlier instruction –Dependence may be for a data value  data hazard –Dependence may be for the next instruction’s address  control hazard (branches, exceptions)

13. 2/3/2009CS152-Spring’09 13 Data Hazards... r1 r0 + 10 r4 r1 + 17... r1 is stale. Oops! r1 … r4 r1… IR 31 PC A B Y R MD1 MD2 addr inst Inst Memory 0x4 Add IR Imm Ext ALU rd1 GPRs rs1 rs2 ws wd rd2 we wdata addr wdata rdata Data Memory we

14. 2/3/2009CS152-Spring’09 14 CS152 Administrivia PS 1 due Tuesday Feb 10 in class Section covering PS 1 on Wednesday Feb 11 –Room/time TBD First Quiz on Thursday Feb 12 –In class, closed-book, no computers or calculators –Covers lectures 1-5 (this week’s lectures) Lecture 7, Tuesday Feb 17 in 320 Soda Lecture 8, Thursday Feb 19 back in 306 Soda See website for full schedule

15. 2/3/2009CS152-Spring’09 15 Resolving Data Hazards (1) Strategy 1: Wait for the result to be available by freezing earlier pipeline stages  interlocks

16. 2/3/2009CS152-Spring’09 16 Feedback to Resolve Hazards Later stages provide dependence information to earlier stages which can stall (or kill) instructions FB 1 stage 1 stage 2 stage 3 stage 4 FB 2 FB 3 FB 4 Controlling a pipeline in this manner works provided the instruction at stage i+1 can complete without any interference from instructions in stages 1 to i (otherwise deadlocks may occur)

17. 2/3/2009CS152-Spring’09 17 IR 31 PC A B Y R MD1 MD2 addr inst Inst Memory 0x4 Add IR Imm Ext ALU rd1 GPRs rs1 rs2 ws wd rd2 we wdata addr wdata rdata Data Memory we nop Interlocks to resolve Data Hazards... r1 r0 + 10 r4 r1 + 17... Stall Condition

18. 2/3/2009CS152-Spring’09 18 stalled stages time t0t1t2t3t4t5t6t7.... IFI 1 I 2 I 3 I 3 I 3 I 3 I 4 I 5 IDI 1 I 2 I 2 I 2 I 2 I 3 I 4 I 5 EX I 1 nopnopnopI 2 I 3 I 4 I 5 MA I 1 nopnopnopI 2 I 3 I 4 I 5 WB I 1 nopnopnopI 2 I 3 I 4 I 5 Stalled Stages and Pipeline Bubbles time t0t1t2t3t4t5t6t7.... (I 1 ) r1 (r0) + 10IF 1 ID 1 EX 1 MA 1 WB 1 (I 2 ) r4 (r1) + 17IF 2 ID 2 ID 2 ID 2 ID 2 EX 2 MA 2 WB 2 (I 3 )IF 3 IF 3 IF 3 IF 3 ID 3 EX 3 MA 3 WB 3 (I 4 ) IF 4 ID 4 EX 4 MA 4 WB 4 (I 5 ) IF 5 ID 5 EX 5 MA 5 WB 5 Resource Usage nop  pipeline bubble

19. 2/3/2009CS152-Spring’09 19 IR 31 PC A B Y R MD1 MD2 addr inst Inst Memory 0x4 Add IR Imm Ext ALU rd1 GPRs rs1 rs2 ws wd rd2 we wdata addr wdata rdata Data Memory we nop Interlock Control Logic Compare the source registers of the instruction in the decode stage with the destination register of the uncommitted instructions. stall C stall ws rs rt ?

20. 2/3/2009CS152-Spring’09 20 C dest Interlock Control Logic ignoring jumps & branches Should we always stall if the rs field matches some rd? IR PC A B Y R MD1 MD2 addr inst Inst Memory 0x4 Add IR Imm Ext ALU rd1 GPRs rs1 rs2 ws wd rd2 we wdata addr wdata rdata Data Memory we 31 nop stall C stall ws rs rt ? we re1re2 C re wswe ws C dest we not every instruction writes a register we not every instruction reads a register re

21. 2/3/2009CS152-Spring’09 21 Source & Destination Registers source(s) destination ALUrd (rs) func (rt) rs, rtrd ALUirt (rs) op immrs rt LWrt M [(rs) + imm] rs rt SWM [(rs) + imm] (rt) rs, rt BZcond (rs) true:PC (PC) + immrs false: PC (PC) + 4rs JPC (PC) + imm JALr31 (PC), PC (PC) + imm31 JRPC (rs) rs JALRr31 (PC), PC (rs) rs31 R-type: op rs rt rd func I-type: op rs rt immediate16 J-type: op immediate26

22. 2/3/2009CS152-Spring’09 22 Deriving the Stall Signal C dest ws = Case opcode ALUrd ALUi, LWrt JAL, JALRR31 we = Case opcode ALU, ALUi, LW (ws  0) JAL, JALR on... off C re re1 = Case opcode ALU, ALUi, on off re2 = Case opcode on off LW, SW, BZ, JR, JALR J, JAL ALU, SW... C stall stall = ((rs D =ws E ).we E + (rs D =ws M ).we M + (rs D =ws W ).we W ). re1 D + ((rt D =ws E ).we E + (rt D =ws M ).we M + (rt D =ws W ).we W ). re2 D This is not the full story !

23. 2/3/2009CS152-Spring’09 23 Hazards due to Loads & Stores... M[(r1)+7]  (r2) r4  M[(r3)+5]... IR 31 PC A B Y R MD1 MD2 addr inst Inst Memory 0x4 Add IR Imm Ext ALU rd1 GPRs rs1 rs2 ws wd rd2 we wdata addr wdata rdata Data Memory we nop Stall Condition Is there any possible data hazard in this instruction sequence? What if (r1)+7 = (r3)+5 ?

24. 2/3/2009CS152-Spring’09 24 Load & Store Hazards However, the hazard is avoided because our memory system completes writes in one cycle ! Load/Store hazards are sometimes resolved in the pipeline and sometimes in the memory system itself. More on this later in the course.... M[(r1)+7]  (r2) r4  M[(r3)+5]... (r1)+7 = (r3)+5  data hazard

25. 2/3/2009CS152-Spring’09 25 Resolving Data Hazards (2) Strategy 2: Route data as soon as possible after it is calculated to the earlier pipeline stage  bypass

26. 2/3/2009CS152-Spring’09 26 Bypassing Each stall or kill introduces a bubble in the pipeline CPI > 1 time t0t1t2t3t4t5t6t7.... (I 1 ) r1 r0 + 10IF 1 ID 1 EX 1 MA 1 WB 1 (I 2 ) r4 r1 + 17IF 2 ID 2 ID 2 ID 2 ID 2 EX 2 MA 2 WB 2 (I 3 )IF 3 IF 3 IF 3 IF 3 ID 3 EX 3 MA 3 (I 4 ) stalled stagesIF 4 ID 4 EX 4 (I 5 ) IF 5 ID 5 timet0t1t2t3t4t5t6t7.... (I 1 ) r1 r0 + 10IF 1 ID 1 EX 1 MA 1 WB 1 (I 2 ) r4  r1 + 17IF 2 ID 2 EX 2 MA 2 WB 2 (I 3 )IF 3 ID 3 EX 3 MA 3 WB 3 (I 4 ) IF 4 ID 4 EX 4 MA 4 WB 4 (I 5 ) IF 5 ID 5 EX 5 MA 5 WB 5 A new datapath, i.e., a bypass, can get the data from the output of the ALU to its input

27. 2/3/2009CS152-Spring’09 27 Adding a Bypass ASrc... (I 1 )r1 r0 + 10 (I 2 )r4 r1 + 17 r4 r1r1  IR PC A B Y R MD1 MD2 addr inst Inst Memory 0x4 Add IR Imm Ext ALU rd1 GPRs rs1 rs2 ws wd rd2 we wdata addr wdata rdata Data Memory we 31 nop stall D EMW When does this bypass help? r1 M[r0 + 10] r4 r1 + 17 JAL 500 r4 r31 + 17 yesno

28. 2/3/2009CS152-Spring’09 28 The Bypass Signal Deriving it from the Stall Signal ASrc = (rs D =ws E ).we E.re1 D we = Case opcode ALU, ALUi, LW (ws  0) JAL, JALR on... off No because only ALU and ALUi instructions can benefit from this bypass Is this correct? Split we E into two components: we-bypass, we-stall stall = ( ((rs D =ws E ).we E + (rs D =ws M ).we M + (rs D =ws W ).we W ).re1 D +((rt D =ws E ).we E + (rt D =ws M ).we M + (rt D =ws W ).we W ).re2 D ) ws = Case opcode ALUrd ALUi, LWrt JAL, JALRR31

29. 2/3/2009CS152-Spring’09 29 Bypass and Stall Signals we-bypass E = Case opcode E ALU, ALUi(ws  0)... off ASrc = (rs D =ws E ).we-bypass E. re1 D Split we E into two components: we-bypass, we-stall stall = ((rs D =ws E ).we-stall E + (rs D =ws M ).we M + (rs D =ws W ).we W ). re1 D +((rt D = ws E ).we E + (rt D = ws M ).we M + (rt D = ws W ).we W ). re2 D we-stall E = Case opcode E LW (ws  0) JAL, JALRon... off

30. 2/3/2009CS152-Spring’09 30 Fully Bypassed Datapath ASrc IR PC A B Y R MD1 MD2 addr inst Inst Memory 0x4 Add IR ALU Imm Ext rd1 GPRs rs1 rs2 ws wd rd2 we wdata addr wdata rdata Data Memory we 31 nop stall D EMW PC for JAL,... BSrc Is there still a need for the stall signal ? stall = (rs D =ws E ). (opcode E =LW E ).(ws E 0 ).re1 D + (rt D =ws E ). (opcode E =LW E ).(ws E 0 ).re2 D

31. 2/3/2009CS152-Spring’09 31 Resolving Data Hazards (3) Strategy 3: Speculate on the dependence. Two cases: Guessed correctly  do nothing Guessed incorrectly  kill and restart

32. 2/3/2009CS152-Spring’09 32 Next Time: Control Hazards Branches/Jumps Exceptions/Interrupts

33. 2/3/2009CS152-Spring’09 33 Acknowledgements These slides contain material developed and copyright by: –Arvind (MIT) –Krste Asanovic (MIT/UCB) –Joel Emer (Intel/MIT) –James Hoe (CMU) –John Kubiatowicz (UCB) –David Patterson (UCB) MIT material derived from course 6.823 UCB material derived from course CS252