1. Lecture 15: Pipelining and Hazards CS 2011 Fall 2014, Dr. Rozier

2. MIDTERM II

3. Midterm II November 13 th Plan for remaining time

4. Midterm II Topics Any topic from Midterm I is fair game – Mostly will be important in terms of how they relate to what we have done. New topics – Code analysis – Load/Store – Decision and Control – Branching – The Stack – Integer Arithmetic – Foating Point – Pipelining and Hazards

5. PIPELINING AND HAZARDS

6. What needs to be done to “Process” an Instruction? Check the PC Fetch the instruction from memory Decode the instruction and set control lines appropriately Execute the instruction – Use ALU – Access Memory – Branch Store Results PC = PC + 4, or PC = branch target

7. Pipeline Fetch Decode Issue Integer Multiply Floating Point Load Store Write Back

8. Pipelining Analogy Pipelined laundry: overlapping execution – Parallelism improves performance §4.5 An Overview of Pipelining

9. Hazards What could cause pipelining to fail?

10. Hazards Situations that prevent starting the next instruction in the next cycle Structure hazards – A required resource is busy Data hazard – Need to wait for previous instruction to complete its data read/write Control hazard – Deciding on control action depends on previous instruction

11. Structure Hazards Conflict for use of a resource In pipeline with a single memory – Load/store requires data access – Instruction fetch would have to stall for that cycle Would cause a pipeline “bubble” Hence, pipelined datapaths require separate instruction/data memories – Or separate instruction/data caches

12. Data Hazards An instruction depends on completion of data access by a previous instruction – addr0, r4, r1 subr2, r0, r3

13. Forwarding (aka Bypassing) Use result when it is computed – Don’t wait for it to be stored in a register – Requires extra connections in the datapath

14. Load-Use Data Hazard Can’t always avoid stalls by forwarding – If value not computed when needed – Can’t forward backward in time! ldr r0 [r2,#0] sub r1, r1, r0

15. Code Scheduling to Avoid Stalls Reorder code to avoid use of load result in the next instruction C code for A = B + E; C = B + F; ldrr1, [r0, #0] ldrr2, [r0, #4] addr3, r1, r2 strr3, [r0, #12] ldrr4, [r0, #8] addr5, r1, r4 strr5, [0, #16] stall ldrr1, [r0, #0] ldrr2, [r0, #4] ldrr4, [r0, #8] addr3, r1, r2 strr3, [r0, #12] addr5, r1, r4 strr5, [0, #16] ___________________

16. Control Hazards Branch determines flow of control – Fetching next instruction depends on branch outcome – Pipeline can’t always fetch correct instruction Still working on ID stage of branch In pipeline – Need to compare registers and compute target early in the pipeline – Add hardware to do it in ID stage

17. Stall on Branch Wait until branch outcome determined before fetching next instruction adds r4, r5, r6 beq label add r7, r8, r9

18. Branch Prediction Longer pipelines can’t readily determine branch outcome early – Stall penalty becomes unacceptable Predict outcome of branch – Only stall if prediction is wrong In pipeline – Can predict branches not taken – Fetch instruction after branch, with no delay

19. MIPS with Predict Not Taken Prediction correct Prediction incorrect adds r4, r5, r6 beq label add r7, r8, r9 adds r4, r5, r6 beq label sub r7, r8, r9

20. More-Realistic Branch Prediction Static branch prediction – Based on typical branch behavior – Example: loop and if-statement branches Predict backward branches taken Predict forward branches not taken Dynamic branch prediction – Hardware measures actual branch behavior e.g., record recent history of each branch – Assume future behavior will continue the trend When wrong, stall while re-fetching, and update history

21. Pipeline Summary Pipelining improves performance by increasing instruction throughput – Executes multiple instructions in parallel – Each instruction has the same latency Subject to hazards – Structure, data, control Instruction set design affects complexity of pipeline implementation The BIG Picture

22. MIPS Pipelined Datapath §4.6 Pipelined Datapath and Control WB MEM Right-to-left flow leads to hazards

23. Pipeline registers Need registers between stages – To hold information produced in previous cycle

24. Pipeline Operation Cycle-by-cycle flow of instructions through the pipelined datapath – “Single-clock-cycle” pipeline diagram Shows pipeline usage in a single cycle Highlight resources used – c.f. “multi-clock-cycle” diagram Graph of operation over time We’ll look at “single-clock-cycle” diagrams for load & store

25. IF for Load, Store, …

26. ID for Load, Store, …

27. EX for Load

28. MEM for Load

29. WB for Load Wrong register number

30. Corrected Datapath for Load

31. EX for Store

32. MEM for Store

33. WB for Store

34. Multi-Cycle Pipeline Diagram Form showing resource usage

35. Multi-Cycle Pipeline Diagram Traditional form

36. Single-Cycle Pipeline Diagram State of pipeline in a given cycle

37. Data Hazards in ALU Instructions Consider this sequence: sub r2, r1, r3 and r7, r2, r5 or r8, r6, r2 add r9, r2, r2 str r10, [r2, #100] We can resolve hazards with forwarding – How do we detect when to forward?

38. Dependencies & Forwarding sub r2, r1, r3 and r7, r2, r5 or r8, r6, r2 add r9, r2, r2 str r10, [r2, #100]

39. Detecting the Need to Forward Pass register numbers along pipeline – e.g., ID/EX.RegisterRs = register number for Rs sitting in ID/EX pipeline register ALU operand register numbers in EX stage are given by – ID/EX.RegisterRs, ID/EX.RegisterRt Data hazards when 1a. EX/MEM.RegisterRd = ID/EX.RegisterRs 1b. EX/MEM.RegisterRd = ID/EX.RegisterRt 2a. MEM/WB.RegisterRd = ID/EX.RegisterRs 2b. MEM/WB.RegisterRd = ID/EX.RegisterRt Fwd from EX/MEM pipeline reg Fwd from MEM/WB pipeline reg

40. Detecting the Need to Forward But only if forwarding instruction will write to a register! – EX/MEM.RegWrite, MEM/WB.RegWrite

41. Forwarding Paths

42. Double Data Hazard Consider the sequence: add r1,r1,r2 add r1,r1,r3 add r1,r1,r4 Both hazards occur – Want to use the most recent Revise MEM hazard condition – Only fwd if EX hazard condition isn’t true

43. Chapter 4 — The Processor — 43 Datapath with Forwarding

44. Load-Use Data Hazard Need to stall for one cycle ldr r2, [r1, #20] and r4, r2, r5 or r8, r2, r6 and r9, r4, r2 add r1, r6, r7

45. Stall/Bubble in the Pipeline Stall inserted here ldr r2, [r1, #20] and becomes nop and r4, r2, r5 or r8, r2, r6 add r9, r4, r2

46. Datapath with Hazard Detection

47. Stalls and Performance Stalls reduce performance – But are required to get correct results Compiler can arrange code to avoid hazards and stalls – Requires knowledge of the pipeline structure The BIG Picture

48. Pitfalls Poor ISA design can make pipelining harder – e.g., complex instruction sets (VAX, IA-32) Significant overhead to make pipelining work IA-32 micro-op approach – e.g., complex addressing modes Register update side effects, memory indirection – e.g., delayed branches Advanced pipelines have long delay slots

49. Concluding Remarks ISA influences design of datapath and control Datapath and control influence design of ISA Pipelining improves instruction throughput using parallelism – More instructions completed per second – Latency for each instruction not reduced Hazards: structural, data, control

50. WRAP UP

51. For next time Exam Review