1. Review: Pipelining

2. Pipelining Laundry Example Ann, Brian, Cathy, Dave each have one load of clothes to wash, dry, and fold Washer takes 30 minutes Dryer takes 40 minutes “Folder” takes 20 minutes ABCD

3. Sequential Laundry Sequential laundry takes 6 hours for 4 loads ABCD 304020304020304020304020 6 PM 789 10 11 Midnight TaskOrderTaskOrder Time

4. Pipelined Laundry Start work ASAP Pipelined laundry takes 3.5 hours for 4 loads ABCD 6 PM 789 10 11 Midnight TaskOrderTaskOrder Time 3040 20

5. Pipelining: Observations Multiple tasks operating simultaneously Pipelining doesn’t help latency of single task, it helps throughput of entire workload Pipeline rate limited by slowest pipeline stage Potential speedup = Number pipe stages Unbalanced lengths of pipe stages reduces speedup Time to “fill” pipeline and time to “drain” it reduces speedup ABCD 6 PM 789 TaskOrderTaskOrder Time 3040 20

6. 5 Steps of DLX Datapath Figure 3.1 Memory Access Write Back Instruction Fetch Instr. Decode Reg. Fetch Execute Addr. Calc PC + Inst. Mem. 4 NPC IR Regs Imm. Sign Ext. A B MUXMUX MUXMUX MUXMUX MUXMUX LMD ALU Data Mem. ALU Output Zero?Cond. 16 32

7. Pipelined DLX Datapath Figure 3.4 Memory Access Write Back Instruction Fetch Instr. Decode Reg. Fetch Execute Addr. Calc. PC + Inst. Mem. 4 IF/ID Regs Sign Ext. MUXMUX MUXMUX MUXMUX MUXMUX ALU Data Mem. Zero? 1632 ID/EXEX/MEMMEM/WB

8. Visualizing Pipelining Figure 3.3 I n s t r. O r d e r Time (clock cycles)

9. Limits to Pipelining Hazards prevent next instruction from executing during its designated clock cycle –Structural hazards: HW cannot support this combination of instructions –Data hazards: Instruction depends on result of prior instruction still in the pipeline –Control hazards: Pipelining of branches & other instructions that change the PC Common solution is to stall the pipeline until the hazard is resolved, inserting one or more “bubbles” in the pipeline

10. One Memory Port/Structural Hazards Figure 3.6 I n s t r. O r d e r Time (clock cycles) Load Instr 1 Instr 2 Instr 3 Instr 4

11. One Memory Port/Structural Hazards Figure 3.7 I n s t r. O r d e r Load Instr 1 Instr 2 stall Instr 3

12. Speed Up Equation for Pipelining Speedup from pipelining = Ave Instr Time unpipelined Ave Instr Time pipelined = CPI unpipelined x Clock Cycle unpipelined CPI pipelined x Clock Cycle pipelined = CPI unpipelined Clock Cycle unpipelined CPI pipelined Clock Cycle pipelined Ideal CPI = CPI unpipelined /Pipeline depth Speedup = Ideal CPI x Pipeline depth Clock Cycle unpipelined CPI pipelined Clock Cycle pipelined x x

13. Speed Up Equation for Pipelining CPI pipelined = Ideal CPI + Pipeline stall clock cycles per instr Speedup = Ideal CPI x Pipeline depth Clock Cycle unpipelined Ideal CPI + Pipeline stall CPI Clock Cycle pipelined What is the maximum possible speedup? Speedup = Pipeline depth Clock Cycle unpipelined 1 + Pipeline stall CPI Clock Cycle pipelined x x

14. Example: Dual-port vs. Single-port Machine A: Dual ported memory Machine B: Single ported memory, but its pipelined implementation has a 1.05 times faster clock rate Ideal CPI = 1 for both Loads are 40% of instructions executed SpeedUp A = Pipeline Depth/(1 + 0) x (clock unpipe /clock pipe ) = Pipeline Depth SpeedUp B = Pipeline Depth/(1 + 0.4) x (clock unpipe /(clock pipe ) = (Pipeline Depth/1.4) x 1.05 = 0.75 x Pipeline Depth SpeedUp A / SpeedUp B = Pipeline Depth/(0.75 x Pipeline Depth) = 1.33 Machine A is 1.33 times faster

15. Data Hazard on R1 Figure 3.9

16. Three Generic Data Hazards Instr I followed by Instr J Read After Write (RAW) Instr J tries to read operand before Instr I writes it

17. Three Generic Data Hazards Instr I followed by Instr J Write After Read (WAR) Instr J tries to write operand before Instr I reads it Can’t happen in DLX 5 stage pipeline because: – All instructions take 5 stages, – Reads are always in stage 2, and – Writes are always in stage 5

18. Three Generic Data Hazards Instr I followed by Instr J Write After Write (WAW) Instr J tries to write operand before Instr I writes it – Leaves wrong result ( Instr I not Instr J ) Can’t happen in DLX 5 stage pipeline because: – All instructions take 5 stages, and – Writes are always in stage 5 Will see WAR and WAW in later more complicated pipes

19. Forwarding to Avoid Data Hazard Figure 3.10

20. HW Change for Forwarding Figure 3.20

21. Data Hazard Even with Forwarding Figure 3.12

22. Data Hazard Even with Forwarding Figure 3.13

23. Try producing fast code for a = b + c; d = e – f; assuming a, b, c, d,e, and f in memory. Slow code: LW Rb,b LW Rc,c ADD Ra,Rb,Rc SW a,Ra LW Re,e LW Rf,f SUB Rd,Re,Rf SWd,Rd Software Scheduling to Avoid Load Hazards Fast code: LW Rb,b LW Rc,c LW Re,e ADD Ra,Rb,Rc LW Rf,f SW a,Ra SUB Rd,Re,Rf SWd,Rd

24. Compiler Avoiding Load Stalls % loads stalling pipeline 0%20%40%60%80% tex spice gcc 25% 14% 31% 65% 42% 54% scheduledunscheduled

25. Control Hazard on Branches Three Stage Stall

26. Branch Stall Impact If CPI = 1, if 30% branch, Stall 3 cycles => new CPI = 1.9! Two part solution: –Determine branch taken or not sooner, AND –Compute taken branch address earlier DLX branch tests if register = 0 or <> 0 DLX Solution: –Move Zero test to ID/RF stage –Adder to calculate new PC in ID/RF stage –1 clock cycle penalty for branch versus 3

27. Pipelined DLX Datapath Figure 3.22

28. Four Branch Hazard Alternatives #1: Stall until branch direction is clear #2: Predict Branch Not Taken –Execute successor instructions in sequence –“Squash” instructions in pipeline if branch actually taken –Advantage of late pipeline state update –47% DLX branches not taken on average –PC+4 already calculated, so use it to get next instruction #3: Predict Branch Taken –53% DLX branches taken on average –But haven’t calculated branch target address in DLX »DLX still incurs 1 cycle branch penalty

29. Four Branch Hazard Alternatives #4: Delayed Branch –Define branch to take place AFTER a following instruction branch instruction sequential successor 1 sequential successor 2........ sequential successor n branch target if taken –1 slot delay allows proper decision and branch target address in 5 stage pipeline –DLX uses this Branch delay of length n

30. Delayed Branch Where to get instructions to fill branch delay slot? –Before branch instruction –From the target address: only valuable when branch taken –From fall through: only valuable when branch not taken Compiler effectiveness for single branch delay slot: –Fills about 60% of branch delay slots –About 80% of instructions executed in branch delay slots useful in computation –About 50% (60% x 80%) of slots usefully filled

31. Pipelining Summary Just overlap tasks, and easy if tasks are independent Speed Up vs Pipeline Depth: Hazards limit performance on computers: –Structural: need more HW resources –Data: need forwarding, compiler scheduling –Control: discuss next time Speedup = Pipeline Depth 1 + Pipeline stall CPI X Clock Cycle Unpipelined Clock Cycle Pipelined