1. Slide 1 Instruction-Level Parallelism Review of Pipelining (the laundry analogy)

2. Slide 2 Instruction-Level Parallelism Review of Pipelining (Appendix A)

3. Slide 3 Instruction-Level Parallelism Review of Pipelining (Appendix A) –MIPS pipeline –MIPS pipeline five stages : »IF – instruction fetch »ID – instruction decoding and operands fetch »EX – execution using ALU, including effective address and target address computing »MEM – accessing memory for L & S instructions »WB – write result back to (destination) register

4. Slide 4 The “naïve” MIPS pipeline

5. Slide 5 The “naïve” MIPS pipeline -- implementation Instruction-Level Parallelism

6. Slide 6 Instruction-Level Parallelism A series of datapaths shifted in time

7. Slide 7 Instruction-Level Parallelism A pipeline showing the pipeline registers between stages

8. Slide 8 The major hurdles of pipelining: pipeline hazards Structural HazardsStructural Hazards: resource conflicts, such as bus, register file ports, memory ports, etc.

9. Slide 9 The major hurdles of pipelining: pipeline hazards Data Hazards forwardingData Hazards: data dependency (producer-consumer relationship, or read after write). Some can be resolved by forwarding

10. Slide 10 The major hurdles of pipelining: pipeline hazards Data HazardsData Hazards: data hazards detection in MIPS pipeline

11. Slide 11 The major hurdles of pipelining: pipeline hazards Data HazardsData Hazards: the logic for forwarding of data in MIPS pipeline

12. Slide 12 The major hurdles of pipelining: pipeline hazards Data HazardsData Hazards: the forwarding of data in MIPS pipeline

13. Slide 13 The major hurdles of pipelining: pipeline hazards Data Hazards forwarding,Data Hazards: Some cannot be resolved by forwarding, thus requiring stalls

14. Slide 14 The major hurdles of pipelining: pipeline hazards Data HazardsData Hazards: Avoid non-forwardable data hazards through compiler scheduling:

15. Slide 15 The major hurdles of pipelining: pipeline hazards Branch (Control) HazardsBranch (Control) Hazards: can cause greater performance loss (e.g., a 3-cycle loss in the “naïve” MIPS pipeline)

16. Slide 16 The major hurdles of pipelining: pipeline hazards Branch (Control) HazardsBranch (Control) Hazards: improved MIPS pipelined with one- cycle loss

17. Slide 17 The major hurdles of pipelining: pipeline hazards Reducing branch penaltiesReducing branch penalties : 1.Freeze or Flussh 2.Predict-not-taken or Predict-taken 3.Delayed Branch 1)Branch instruction 2)Sequential successor 3)Branch target if taken “Canceling/nullifying” Branch if prediction incorrect

18. Slide 18 The major hurdles of pipelining: pipeline hazards Scheduling the branch delay slotScheduling the branch delay slot :

19. Slide 19 Performance of Pipelining Example 1Example 1 : –Consider an unpipelined machine A and a pipelined machine B where CCT A = 10ns, CPI(A) ALU = CPI(A) Br = 4, CPI(A) l/s = 5, CCT B = 11ns. Assuming an instruction mix of 40% for ALU, 20% for branches, and 40% for l/s, what is the speedup of B over A under ideal conditions?machine A machine B

20. Slide 20 Performance of Pipelining Impacts of pipeline hazardsImpacts of pipeline hazards :

21. Slide 21 Performance of Pipelining Performance of branch schemesPerformance of branch schemes : Overall costs of a variety of branch schemes with the MIPS pipeline

22. Slide 22 Performance of Pipelining Example 2Example 2: For a deeper pipeline such as that in a MIPS R4000, it takes three pipeline stages before the target-address is known and an additional stage before the condition is evaluated. This leads to the branch penalties for the three simplest branch schemes listed below:R4000 Find the effective addition to the CPI arising from branches for this pipeline, assuming that unconditional, untaken conditional, and taken conditional branches account for 4%, 6%, and 10%, respectively.Answer:

23. Slide 23 What Makes Pipelining Hard to Implement? 1.Exceptional conditions (e.g., interrupts, etc) often change the order of instruction execution;

24. Slide 24 What Makes Pipelining Hard to Implement? Actions needed for different types of exceptional conditions:

25. Slide 25 What Makes Pipelining Hard to Implement? Stopping and Restarting Execution: Two Challenges

26. Slide 26 What Makes Pipelining Hard to Implement? Stopping and Restarting Execution: Two Challenges (cont’d)

27. Slide 27 What Makes Pipelining Hard to Implement? Precise Exception Handling in MIPS Pipeline StageProblem exceptions occurring IF Page fault on instruction fetch; misaligned memory access; memory protection violation IDUndefined or illegal opcode EXArithmetic exception MEMPage fault on data fetch; misaligned memory access; memory-protection violation WBNone

28. Slide 28 What Makes Pipelining Hard to Implement? Precise Exception Handling in MIPS

29. Slide 29 Extending MIPS Pipeline to Handle Multicycle Operations Handle floating point operations: single cycle (CPI=1)  very long CCT or highly complex logic circuit Multiple cycle  long latency: with EX cycle repeated many times and/or with multiple PF function units The MIPS pipeline with three additional unpipelined, floating point units

30. Slide 30 Extending MIPS Pipeline to Handle Multicycle Operations Pipelining FP functional units: LatencyLatency: number of intervening cycles between the producer and the consumer of an operand -- 0 for ALU and 1 for LW Initiation intervalInitiation interval: number of minimum cycles between two issues of instructions using the same functional unit. F. UnitInt. ALUData MemFP AddMultiplyDivide Latency 013624 Init. Interval 111125

31. Slide 31 Extending MIPS Pipeline to Handle Multicycle Operations Pipeline timing of a set of independent FP instructions : A typical FP code sequence showing the stalls arising from RAW hazards : Three instructions want to perform a write back to the FP register simultaneously MUL.D IFIDM1M2M3M4M5M6M7MEMWB ADD.D IFIDA1A2A3A4MEMWB L.D IFIDEXMEMWB S.D IFIDEXMEMWB 1234567891011121314151617 L.D. F4,0(R2) IFIDEXMMWB MUL.D F0,F4,F6 IFIDStallM1M2M3M4M5M6M7MMWB ADD.D F2,F0,F8 IFSt’lIDSt’l A1A2A3A4MMWB S.D. F2,0(R2) IFSt’l IDEXSt’l MM 12 3 4567891011 MUL.D F0, F4, F6IFIDM1M2M3M4M5M6M7MEMWB …IFIDEXMEMWB …IFIDEXMEMWB ADD.D F2, F4, F6IFIDA1A2A3A4MEMWB …IFIDEXMEMWB …IFIDEXMEMWB L.D. F2, 0(R2)IFIDEXMEMWB

32. Slide 32 Extending MIPS Pipeline to Handle Multicycle Operations Difficulties in exploiting ILP:Difficulties in exploiting ILP: various hazards that impose dependency among instructions, as a result: –RAW(read after write): j tries to read a source before i writes to it –WAW(write after write): j tries to write an operand before it is written by i –WAR(write after read): j tries to write a destination before it is read by Implementing pipeline in FP: hazards and forwarding in longer latency pipelinesImplementing pipeline in FP: hazards and forwarding in longer latency pipelines –Divide not fully pipelined (structural hazard) –Multiple –Multiple writes in a cycle and arrive at WB variably,WAW and structural hazards. Would there be WAR? –Out-of-order completion of instructions  more problems for exception handling –Higher RAW frequency and longer stalls due to longer latency

33. Slide 33 Extending MIPS Pipeline to Handle Multicycle Operations Introduce interlock:Introduce interlock: –tracking the use of write port at ID and stalling issue if detected –use shift register for tracking issued instructions' use of write port –stall when entering MEM: 1.can stall any of the contending instructions, 2.no need to detect conflict early when is it harder to see, 3.give priority to the unit with the longest latency, 4.can cause bottleneck stalling –WAW occurs if LD is issued one cycle earlier and has F2 as destination (WAW with ADDD); Solution: 1.delay issuing LD until ADDD enters MEM, or, 2.stamp out result of ADD »Hazard detection with FP pipeline: 1.check for structural hazards: a. functional units, b. write ports 2.check for RAW hazard: source reg. in ID = dest. reg. (issued) 3.check for WAW hazard: dest reg. in ID = dest. reg. (issued)

34. Slide 34 Extending MIPS Pipeline to Handle Multicycle Operations Maintain precise exception:Maintain precise exception: –Example of out-of-order completion: »DIVF F0, F2, F3 ; exception of SUBF at end of ADDF »ADDF F10, F10, F8 ; cause imprecise exception which »SUBF F12, F12, F14 ; cannot be solved by HW/SW –Solutions: 1.Fast imprecise (tolerable in 60's & 70s, but much less so now due to pipelined FP, virtual memory, and IEEE standard) or slow precise 2.Buffering of result until all predecessors finish: –the bigger the difference among instruction execution lengths, the more expensive to implement (e.g., large number of comparators and MUXs and large amount of buffer space) –history file: keeps track of register values – future file: keeps newer values of registers until all predecessors are completed 3.Quasi-precise exception: keep enough information for trap-handling routine to create a precise sequence for exception: –operations in the pipeline and their PCs –software finishes all instructions issued prior to the latest completed instruction 4.Guarded issuing: issue only if it is certain that all prior instructions will complete without causing an exception –stalling to maintain precise exception

35. Slide 35 The MIPS R4000 Pipeline –R4000 pipeline leads to a 2-cycle load delay

36. Slide 36 The MIPS R4000 Pipeline –R4000 pipeline leads to a 3-cycle basic branch delay since the condition evaluation is performed during the EX stage

37. Slide 37 Dynamic Scheduling with Scoreboard Dynamic Scheduling: Dynamic Scheduling: hardware re-arranges the instruction execution order to reduce stalls: 1.handles situations where dependences are unknown or difficult to detect at compile time, thus simplifying the compiler design; 2.increases portability of the compiled code; 3.solves problems associated with the so-called “head-of-the-queue” (HOTQ) blocking caused by “in-order issue” of earlier pipelines. Example: 4.MIPS, which is “in-order issue”, can be made to “out-of-order” execute (implying “out-of-order” completion) by splitting ID into two phases: (1) In-order Issue: check for structural hazards. (2) Read operands: wait until no data hazards, then read operands (and then execute, possibly out-of-order!). The HOTQ problem above can be solved in this new MIPS!

38. Slide 38 Dynamic Scheduling with Scoreboard

39. Slide 39 Dynamic Scheduling with Scoreboard

40. Slide 40 Dynamic Scheduling with Scoreboard

41. Slide 41 Dynamic Scheduling with Scoreboard

42. Slide 42 Dynamic Scheduling with Scoreboard

43. Slide 43 Dynamic Scheduling with Scoreboard

44. Slide 44 Dynamic Scheduling with Scoreboard

45. Slide 45 Unpipelined Processor (MIPS)

46. Slide 46 Pipelined Processor (MIPS)

47. Slide 47 The Eight-stage Pipeline of the R4000

48. Slide 48 A 2-cycle Load Delay of The R4000 Integer Pipeline