Prof. John Nestor ECE Department Lafayette College Easton, Pennsylvania ECE Computer Organization Pipelined Processor Design 2 Feb 2004 Reading: , 6.8 Portions of these slides are derived from: Textbook figures © 1998 Morgan Kaufmann Publishers all rights reserved Tod Amon's COD2e Slides © 1998 Morgan Kaufmann Publishers all rights reserved Dave Patterson’s CS 152 Slides - Fall 1997 © UCB Rob Rutenbar’s Slides - Fall 1999 CMU other sources as noted

Feb 2005Pipelining 22 Pipelining Outline  Introduction  Pipelined Processor Design   Datapath  Control  Dealing with Hazards & Forwarding  Branch Prediction  Exceptions  Performance  Advanced Pipelining  Superscalar  Dynamic Pipelining  Examples

Feb 2005Pipelining 23 Pipelining in MIPS  MIPS architecture was designed to be pipelined  Simple instruction format (makes IF, ID easy) Single-word instructions Small number of instruction formats Common fields in same place (e.g., rs, rt) in different formats  Memory operations only in lw, sw instructions (simplifies EX)  Memory operands aligned in memory (simplifies MEM)  Single value for writeback (limits forwarding)  Pipelining is harder in CISC architectures

Feb 2005Pipelining 24 Pipelined Datapath with Control Signals

Feb 2005Pipelining 25 Next Step: Adding Control  Basic approach: build on single-cycle control  Place control unit in ID stage  Pass control signals to following stages  Later: extra features to deal with:  Data forwarding  Stalls  Exceptions

Feb 2005Pipelining 26 Control for Pipelined Datapath Source: Book Fig. 6.29, p 469 RegDst ALUOp[1:0] ALUSrc MemRead MemWrite Branch RegWrite MemtoReg

Feb 2005Pipelining 27 Control for Pipelined Datapath Source: Book Fig. 6.25, p 401

Feb 2005Pipelining 28 Datapath and Control Unit

Feb 2005Pipelining 29 Tracking Control Signals - Cycle 1 LW

Feb 2005Pipelining 210 Tracking Control Signals - Cycle 2 SWLW

Feb 2005Pipelining 211 Tracking Control Signals - Cycle 3 ADDSWLW

Feb 2005Pipelining 212 Tracking Control Signals - Cycle 4 SUBADD SW LW 1 0 0

Feb 2005Pipelining ADD Tracking Control Signals - Cycle 5 SUB SW LW

Feb 2005Pipelining 214 Pipelining Outline - Coming Up  Introduction  Pipelined Processor Design  Datapath  Control  Dealing with Hazards & Forwarding   Branch Prediction  Exceptions  Performance  Advanced Pipelining  Superscalar  Dynamic Pipelining  Examples

Feb 2005Pipelining 215 Data Hazards Revisited…  Data hazards occur when data is used before it is stored (Fig. 6.28)

Feb 2005Pipelining 216 Data Hazard Solution: Forwarding  Key idea: connect data internally before it's stored (Fig. 6.29)

Feb 2005Pipelining 217 Data Hazard Solution: Forwarding  Add hardware to feed back ALU and MEM results to both ALU inputs (Fig. 6.32)

Feb 2005Pipelining 218 Controlling Forwarding  Need to test when register numbers match in rs, rt, and rd fields stored in pipeline registers  "EX" hazard:  EX/MEM - test whether instruction writes register file and examine rd register  ID/EX - test whether instruction reads rs or rt register and matches rd register in EX/MEM  "MEM" hazard:  MEM/WB - test whether instruction writes register file and examine rd (rt) register  ID/EX - test whether instruction reads rs or rt register and matches rd (rt) register in EX/MEM

Feb 2005Pipelining 219 Forwarding Unit Detail - EX Hazard if (EX/MEM.RegWrite and (EX/MEM.RegisterRd ≠ 0) and (EX/MEM.RegisterRd = ID/EX.RegisterRs)) ForwardA = 10 if (EX/MEM.RegWrite and (EX/MEM.RegisterRd ≠ 0) and (EX/MEM.RegisterRd = ID/EX.RegisterRt)) ForwardB = 10

Feb 2005Pipelining 220 Forwarding Unit Detail - MEM Hazard if (MEM/WB.RegWrite and (MEM/WB.RegisterRd ≠ 0) and (MEM/WB.RegisterRd = ID/EX.RegisterRs)) ForwardA = 01 if (MEM/WB.RegWrite and (MEM/WB.RegisterRd ≠ 0) and (MEM/WB.RegisterRd = ID/EX.RegisterRt)) ForwardB = 01

Feb 2005Pipelining 221 EX Hazard Complication  What if we a register is changed more than once?  add $1, $1, $2;  add $1, $1, $3;  add $1, $1, $4;  Answer: forward most recent result (in MEM stage)

Feb 2005Pipelining 222 Forwarding Unit Detail - MEM Hazard Revised if (MEM/WB.RegWrite and (MEM/WB.RegisterRd ≠ 0) and (EX/MEM.RegisterRd ≠ ID/EX.RegisterRs) and (MEM/WB.RegisterRd = ID/EX.RegisterRs)) ForwardA = 01 if (MEM/WB.RegWrite and (MEM/WB.RegisterRd ≠ 0) and (EX/MEM.RegisterRd ≠ ID/EX.RegisterRt) and (MEM/WB.RegisterRd = ID/EX.RegisterRt)) ForwardB = 01

Feb 2005Pipelining 223 Fig (6.33) Forwarding Elaboration  Extra 2-1 mux needed for immediate instructions Added Mux

Feb 2005Pipelining 224 Data Hazards and Stalls  We still have to stall when register is loaded from memory and used in following instruction (Fig. 6.34)

Feb 2005Pipelining 225 Data Hazards and Stalls  Add a hazard detection unit to detect this condition and stall (Fig. 6.35) Typo: Should read AND

Feb 2005Pipelining 226 Pipelined Processor with Hazard Detection (Fig. 6.36)

Feb 2005Pipelining 227 Hazard Detection Unit - Control Detail if (ID/EX.MemRead and ((ID/EX.RegisterRt = IF/ID.RegisterRs) or ((ID/EX.RegisterRt = IF/ID.RegisterRt))) stall

Feb 2005Pipelining 228 Hazard detection unit - what happens  MUX zeros out control signals for instruction in ID  "squashes” the instruction  “no-op” propagates through following stages  IF/ID holds stalled instruction until next clock cycle  PC holds current value until next clock cycle (re- loads first instruction)

Feb 2005Pipelining 229 Branch Hazards  Just stalling for each branch is not practical  Common assumption: branch not taken  When assumption fails: flush three instructions (Fig. 6.37)

Feb 2005Pipelining 230 Reducing Branch Delay  Key idea: move branch logic to ID stage of pipeline  New adder calculates branch target (PC extend(IMM) << 2)  New hardware tests rs == rt after register read  Add flush signal to squash instruction in IF/ID register  Reduced penalty (1 cycle) when branch taken  Example: Figure 6.38, p. 420

Feb 2005Pipelining 231 Pipelined Processor - Branch Hardware in ID (Old Fig. 6.51)

Feb 2005Pipelining 232 Pipelining Outline  Introduction  Pipelined Processor Design  Datapath  Control  Dealing with Hazards & Forwarding  Branch Prediction   Exceptions  Performance  Advanced Pipelining  Superscalar  Dynamic Pipelining  Examples

Feb 2005Pipelining 233 Branch Prediction  Key idea: instead of always assuming branch not taken, use a prediction based on previous history  Branch history table: small memory index using lower bits instruction address save “what happened” on last execution –branch taken OR –branch not taken  Use history to make prediction

Feb 2005Pipelining 234 More about Branch Prediction  Consider nested loops: for (i=1; I<M; i++) oloop:... for (j=1;j<N; j++) { iloop: } bne $1,$2, iloop } bne $3,$4, oloop  Prediction fails on first and last branch  More history can improve performance

Feb 2005Pipelining 235 Branch Prediction w/2-Bit History  Key idea: must be wrong twice before changing prediction

Feb 2005Pipelining 236 Pipelining Outline  Introduction  Pipelined Processor Design  Datapath  Control  Dealing with Hazards & Forwarding  Branch Prediction  Exceptions   Performance  Advanced Pipelining  Superscalar  Dynamic Pipelining  Examples

Feb 2005Pipelining 237 Pipelining and Exceptions  Exceptions require suspension of execution  Complicating factors  Several instructions are in pipeline  Exception may occur before instruction is complete  Must flush pipeline to suspend execution, but may lose information about the exception

Feb 2005Pipelining 238 Pipelining and Exceptions (cont’d) (Fig. 6.42)

Feb 2005Pipelining 239 Pipelining and Exceptions (cont’d)  Operation: Figure 6.43 (p. 508)  Exceptions make life difficult - take a computer architecture course to learn more.

Feb 2005Pipelining 240 Pipelining Outline  Introduction  Pipelined Processor Design  Datapath  Control  Dealing with Hazards & Forwarding  Branch Prediction  Exceptions  Performance   Advanced Pipelining  Superscalar  Dynamic Pipelining  Examples

Feb 2005Pipelining 241 Performance of the Pipelined Implementation  Use “gcc” instr. mix to calculate CPI lw25%1 cycle (2 cycles when load-use hazard) sw10%1 cycle R-type52%1 cycle branch11%1 cycle (2 when prediction wrong) jump2%2 cycles  Assmptions:  50% of load instructions are followed by immed. use  25% of branch predictions are wrong  Calculating CPI  CPI = (1.5 cycles * 0.25) + (1 cycle * 0.10) + (1 cycle * 0.52) + (1.25 cycles * 0.11) + (2 cycles * 0.02)  CPI = 1.17 cycles per instruction

Feb 2005Pipelining 242 Performance of the Pipelined Implementation (cont’d)  Calculate the average execution time: Pipelined1.17 CPI * 200ps/clock= 234ps Single-Cycle 1 CPI * 600ps/clock=600ps Multicycle4.12 CPI * 200ps / clock=824ps  Speedup of pipelined implementation  2.56X faster than single cycle  3.4X faster than multicycle  “Your mileage may differ” as instruction mix changes

Feb 2005Pipelining 243 Pipelining Outline  Introduction  Pipelined Processor Design  Datapath  Control  Dealing with Hazards & Forwarding  Branch Prediction  Exceptions  Performance  Advanced Pipelining   Superscalar  Dynamic Pipelining  Examples