1. Randal E. Bryant Carnegie Mellon University CS:APP2e CS:APP Chapter 4 Computer Architecture PipelinedImplementation Part II CS:APP Chapter 4 Computer Architecture PipelinedImplementation Part II http://csapp.cs.cmu.edu

2. – 2 – CS:APP2e Overview Make the pipelined processor work! Data Hazards Instruction having register R as source follows shortly after instruction having register R as destination Common condition, don’t want to slow down pipeline Control Hazards Mispredict conditional branch Our design predicts all branches as being taken Naïve pipeline executes two extra instructions Getting return address for ret instruction Naïve pipeline executes three extra instructions Making Sure It Really Works What if multiple special cases happen simultaneously?

3. – 3 – CS:APP2e Pipeline Stages Fetch Select current PC Read instruction Compute incremented PCDecode Read program registersExecute Operate ALUMemory Read or write data memory Write Back Update register file

4. – 4 – CS:APP2e PIPE- Hardware Pipeline registers hold intermediate values from instruction execution Forward (Upward) Paths Values passed from one stage to next Cannot jump past stages e.g., valC passes through decode

5. – 5 – CS:APP2e Data Dependencies: 2 Nop’s

6. – 6 – CS:APP2e Data Dependencies: No Nop 0x000:irmovl$10,%edx 12345678 FDEM W 0x006:irmovl$3,%eax FDEM W FDEMW 0x00c:addl%edx,%eax FDEMW 0x00e: halt # demo-h0.ys E D valA  R[ %edx ]=0 valB  R[ %eax ]=0 D valA  R[ %edx ]=0 valB  R[ %eax ]=0 Cycle 4 Error M M_valE= 10 M_dstE= %edx e_valE  0 + 3 = 3 E_dstE= %eax

7. – 7 – CS:APP2e Stalling for Data Dependencies If instruction follows too closely after one that writes register, slow it down Hold instruction in decode Dynamically inject nop into execute stage 0x000: irmovl $10,%edx 123456789 FDEMW 0x006: irmovl $3,%eax FDEMW 0x00c: nop FDEMW bubble F EMW 0x00e: addl %edx,%eax DDEMW 0x010: halt FDEMW 10 # demo-h2.ys F FDEMW 0x00d: nop 11

8. – 8 – CS:APP2e Stall Condition Source Registers srcA and srcB of current instruction in decode stage Destination Registers dstE and dstM fields Instructions in execute, memory, and write-back stages Special Case Don’t stall for register ID 15 (0xF) Indicates absence of register operand Don’t stall for failed conditional move

9. – 9 – CS:APP2e Detecting Stall Condition 0x000: irmovl $10,%edx 123456789 FDEMW 0x006: irmovl $3,%eax FDEMW 0x00c: nop FDEMW bubble F EMW 0x00e: addl %edx,%eax DDEMW 0x010: halt FDEMW 10 # demo-h2.ys F FDEMW 0x00d: nop 11 Cycle 6 W D W_dstE = %eax W_valE = 3 srcA = %edx srcB = %eax

10. – 10 – CS:APP2e Stalling X3 0x000: irmovl $10,%edx 123456789 FDEMW 0x006: irmovl $3,%eax FDEMW bubble F EMW D EMW 0x00c: addl %edx,%eax DDEMW 0x00e: halt FDEMW 10 # demo-h0.ys FF D F EMW bubble 11 Cycle 4 W W_dstE = %eax D srcA = %edx srcB = %eax M M_dstE = %eax D srcA = %edx srcB = %eax E E_dstE = %eax D srcA = %edx srcB = %eax Cycle 5 Cycle 6

11. – 11 – CS:APP2e What Happens When Stalling? Stalling instruction held back in decode stage Following instruction stays in fetch stage Bubbles injected into execute stage Like dynamically generated nop’s Move through later stages 0x000: irmovl $10,%edx 0x006: irmovl $3,%eax 0x00c: addl %edx,%eax Cycle 4 0x00e: halt 0x000: irmovl $10,%edx 0x006: irmovl $3,%eax 0x00c: addl %edx,%eax # demo-h0.ys 0x00e: halt 0x000: irmovl $10,%edx 0x006: irmovl $3,%eax bubble 0x00c: addl %edx,%eax Cycle 5 0x00e: halt 0x006: irmovl $3,%eax bubble 0x00c: addl %edx,%eax bubble Cycle 6 0x00e: halt bubble 0x00c: addl %edx,%eax bubble Cycle 7 0x00e: halt bubble Cycle 8 0x00c: addl %edx,%eax 0x00e: halt Write Back Memory Execute Decode Fetch

12. – 12 – CS:APP2e Implementing Stalling Pipeline Control Combinational logic detects stall condition Sets mode signals for how pipeline registers should update E M W F D CC rB srcA srcB icodevalEvalMdstEdstM CndicodevalEvalAdstEdstM icodeifunvalCvalAvalBdstEdstMsrcAsrcB valCvalPicodeifunrA predPC d_srcB d_srcA e_Cnd D_icode E_icode M_icode E_dstM Pipe control logic D_bubble D_stall E_bubble F_stall M_bubble W_stall set_cc stat W_stat stat m_stat

13. – 13 – CS:APP2e Pipeline Register Modes Rising clock Rising clock  Output = y yy Rising clock Rising clock  Output = x xx xx n o p Rising clock Rising clock  Output = nop Output = xInput = y stall = 0 bubble = 0 xx Normal Output = xInput = y stall = 1 bubble = 0 xx Stall Output = xInput = y stall = 0 bubble = 1 Bubble

14. – 14 – CS:APP2e Data Forwarding Naïve Pipeline Register isn’t written until completion of write-back stage Source operands read from register file in decode stage Needs to be in register file at start of stageObservation Value generated in execute or memory stageTrick Pass value directly from generating instruction to decode stage Needs to be available at end of decode stage

15. – 15 – CS:APP2e Data Forwarding Example irmovl in write- back stage Destination value in W pipeline register Forward as valB for decode stage

16. – 16 – CS:APP2e Bypass Paths Decode Stage Forwarding logic selects valA and valB Normally from register file Forwarding: get valA or valB from later pipeline stage Forwarding Sources Execute: valE Memory: valE, valM Write back: valE, valM

17. – 17 – CS:APP2e Data Forwarding Example #2 Register %edx Generated by ALU during previous cycle Forward from memory as valA Register %eax Value just generated by ALU Forward from execute as valB

18. – 18 – CS:APP2e Multiple Forwarding Choices Which one should have priority Match serial semantics Use matching value from earliest pipeline stage W R[ %eax ]  3 W R[ %eax ]  1 D valA  R[ %edx ]=10 valB  R[ %eax ]=0 D valA  R[ %edx ]=10 valB  R[ D valA  R[ %eax ]=? valB  0 Cycle 5 W R[ %eax ]  3 M R[ %eax ]  2 W R[ %eax ]  3 E R[ %eax ]  3 Forwarding Priority

19. – 19 – CS:APP2e Implementing Forwarding Add additional feedback paths from E, M, and W pipeline registers into decode stage Create logic blocks to select from multiple sources for valA and valB in decode stage

20. – 20 – CS:APP2e Implementing Forwarding ## What should be the A value? int new_E_valA = [ # Use incremented PC D_icode in { ICALL, IJXX } : D_valP; # Forward valE from execute d_srcA == e_dstE : e_valE; # Forward valM from memory d_srcA == M_dstM : m_valM; # Forward valE from memory d_srcA == M_dstE : M_valE; # Forward valM from write back d_srcA == W_dstM : W_valM; # Forward valE from write back d_srcA == W_dstE : W_valE; # Use value read from register file 1 : d_rvalA; ];

21. – 21 – CS:APP2e Limitation of Forwarding Load-use dependency Value needed by end of decode stage in cycle 7 Value read from memory in memory stage of cycle 8

22. – 22 – CS:APP2e Avoiding Load/Use Hazard Stall using instruction for one cycle Can then pick up loaded value by forwarding from memory stage

23. – 23 – CS:APP2e Detecting Load/Use Hazard ConditionTrigger Load/Use Hazard E_icode in { IMRMOVL, IPOPL } && E_dstM in { d_srcA, d_srcB }

24. – 24 – CS:APP2e Control for Load/Use Hazard Stall instructions in fetch and decode stages Inject bubble into execute stageConditionFDEMW Load/Use Hazard stallstallbubblenormalnormal

25. – 25 – CS:APP2e Branch Misprediction Example Should only execute first 8 instructions 0x000: xorl %eax,%eax 0x002: jne t # Not taken 0x007: irmovl $1, %eax # Fall through 0x00d: nop 0x00e: nop 0x00f: nop 0x010: halt 0x011: t: irmovl $3, %edx # Target (Should not execute) 0x017: irmovl $4, %ecx # Should not execute 0x01d: irmovl $5, %edx # Should not execute demo-j.ys

26. – 26 – CS:APP2e Handling Misprediction Predict branch as taken Fetch 2 instructions at target Cancel when mispredicted Detect branch not-taken in execute stage On following cycle, replace instructions in execute and decode by bubbles No side effects have occurred yet

27. – 27 – CS:APP2e Detecting Mispredicted Branch ConditionTrigger Mispredicted Branch E_icode = IJXX & !e_Cnd

28. – 28 – CS:APP2e Control for Misprediction ConditionFDEMW Mispredicted Branch normalbubblebubblenormalnormal

29. – 29 – CS:APP2e 0x000: irmovl Stack,%esp # Initialize stack pointer 0x006: call p # Procedure call 0x00b: irmovl $5,%esi # Return point 0x011: halt 0x020:.pos 0x20 0x020: p: irmovl $-1,%edi # procedure 0x026: ret 0x027: irmovl $1,%eax # Should not be executed 0x02d: irmovl $2,%ecx # Should not be executed 0x033: irmovl $3,%edx # Should not be executed 0x039: irmovl $4,%ebx # Should not be executed 0x100:.pos 0x100 0x100: Stack: # Stack: Stack pointer Return Example Previously executed three additional instructions demo-retb.ys

30. – 30 – CS:APP2e 0x026: ret FDEM W bubble FDEM W FDEMW FDEMW 0x00b:irmovl$5,%esi# Return FDEMW # demo-retb FDEMW F valC  5 rB  %esi F valC  5 rB  %esi W valM= 0x0b W valM= 0x0b Correct Return Example As ret passes through pipeline, stall at fetch stage While in decode, execute, and memory stage Inject bubble into decode stage Release stall when reach write-back stage

31. – 31 – CS:APP2e Detecting Return ConditionTrigger Processing ret IRET in { D_icode, E_icode, M_icode }

32. – 32 – CS:APP2e 0x026: ret FDEM W bubble FDEM W FDEMW FDEMW 0x00b:irmovl$5,%esi# Return FDEMW # demo-retb FDEMW Control for Return ConditionFDEMW Processing ret stallbubblenormalnormalnormal

33. – 33 – CS:APP2e Special Control Cases Detection Action (on next cycle) ConditionTrigger Processing ret IRET in { D_icode, E_icode, M_icode } Load/Use Hazard E_icode in { IMRMOVL, IPOPL } && E_dstM in { d_srcA, d_srcB } Mispredicted Branch E_icode = IJXX & !e_Cnd ConditionFDEMW Processing ret stallbubblenormalnormalnormal Load/Use Hazard stallstallbubblenormalnormal Mispredicted Branch normalbubblebubblenormalnormal

34. – 34 – CS:APP2e Implementing Pipeline Control Combinational logic generates pipeline control signals Action occurs at start of following cycle E M W F D CC rB srcA srcB icodevalEvalMdstEdstM CndicodevalEvalAdstEdstM icodeifunvalCvalAvalBdstEdstMsrcAsrcB valCvalPicodeifunrA predPC d_srcB d_srcA e_Cnd D_icode E_icode M_icode E_dstM Pipe control logic D_bubble D_stall E_bubble F_stall M_bubble W_stall set_cc stat W_stat stat m_stat

35. – 35 – CS:APP2e Initial Version of Pipeline Control bool F_stall = # Conditions for a load/use hazard E_icode in { IMRMOVL, IPOPL } && E_dstM in { d_srcA, d_srcB } || # Stalling at fetch while ret passes through pipeline IRET in { D_icode, E_icode, M_icode }; bool D_stall = # Conditions for a load/use hazard E_icode in { IMRMOVL, IPOPL } && E_dstM in { d_srcA, d_srcB }; bool D_bubble = # Mispredicted branch (E_icode == IJXX && !e_Cnd) || # Stalling at fetch while ret passes through pipeline IRET in { D_icode, E_icode, M_icode }; bool E_bubble = # Mispredicted branch (E_icode == IJXX && !e_Cnd) || # Load/use hazard E_icode in { IMRMOVL, IPOPL } && E_dstM in { d_srcA, d_srcB };

36. – 36 – CS:APP2e Control Combinations Special cases that can arise on same clock cycle Combination A Not-taken branch ret instruction at branch target Combination B Instruction that reads from memory to %esp Followed by ret instruction

37. – 37 – CS:APP2e Control Combination A Should handle as mispredicted branch Stalls F pipeline register But PC selection logic will be using M_valM anyhow JXX E D M Mispredict JXX E D M Mispredict E ret D M 1 E D M 1 E D M 1 Combination A ConditionFDEMW Processing ret stallbubblenormalnormalnormal Mispredicted Branch normalbubblebubblenormalnormal Combinationstallbubblebubblenormalnormal

38. – 38 – CS:APP2e Control Combination B Would attempt to bubble and stall pipeline register D Signaled by processor as pipeline error Load E Use D M Load/use E ret D M 1 E D M 1 E D M 1 Combination B ConditionFDEMW Processing ret stallbubblenormalnormalnormal Load/Use Hazard stallstallbubblenormalnormal Combinationstall bubble + stall bubblenormalnormal

39. – 39 – CS:APP2e Handling Control Combination B Load/use hazard should get priority ret instruction should be held in decode stage for additional cycle Load E Use D M Load/use E ret D M 1 E D M 1 E D M 1 Combination B ConditionFDEMW Processing ret stallbubblenormalnormalnormal Load/Use Hazard stallstallbubblenormalnormal Combinationstallstallbubblenormalnormal

40. – 40 – CS:APP2e Corrected Pipeline Control Logic Load/use hazard should get priority ret instruction should be held in decode stage for additional cycle ConditionFDEMW Processing ret stallbubblenormalnormalnormal Load/Use Hazard stallstallbubblenormalnormal Combinationstallstallbubblenormalnormal bool D_bubble = # Mispredicted branch (E_icode == IJXX && !e_Cnd) || # Stalling at fetch while ret passes through pipeline IRET in { D_icode, E_icode, M_icode } # but not condition for a load/use hazard && !(E_icode in { IMRMOVL, IPOPL } && E_dstM in { d_srcA, d_srcB });

41. – 41 – CS:APP2e Pipeline Summary Data Hazards Most handled by forwarding No performance penalty Load/use hazard requires one cycle stall Control Hazards Cancel instructions when detect mispredicted branch Two clock cycles wasted Stall fetch stage while ret passes through pipeline Three clock cycles wasted Control Combinations Must analyze carefully First version had subtle bug Only arises with unusual instruction combination