Stalls and flushes Last time, we discussed data hazards that can occur in pipelined CPUs if some instructions depend upon others that are still executing.

Stalls and flushes Last time, we discussed data hazards that can occur in pipelined CPUs if some instructions depend upon others that are still executing. Many hazards can be resolved by forwarding data from the pipeline registers, instead of waiting for the writeback stage. The pipeline continues running at full speed, with one instruction beginning on every clock cycle. Today we’ll see some real limitations of pipelining. Forwarding may not work for data hazards from load instructions. Branches affect the instruction fetch for the next clock cycle. In both of these cases we may need to slow down, or stall, the pipeline. July 10, 2019

Data hazard review A data hazard arises if one instruction needs data that isn’t ready yet. Below, the AND and OR both need to read register $2. But $2 isn’t updated by SUB until the fifth clock cycle. Dependency arrows that point backwards indicate hazards. Clock cycle DM Reg IM sub $2, $1, $3 and $12, $2, $5 or $13, $6, $2 DM Reg IM DM Reg IM July 10, 2019 Stalls and flushes

Forwarding to the rescue!
The desired value ($1 - $3) has actually already been computed—it just hasn’t been written to the registers yet. Forwarding allows other instructions to read ALU results directly from the pipeline registers, without going through the register file. Clock cycle sub $2, $1, $3 and $12, $2, $5 or $13, $6, $2 IM Reg DM Reg IM Reg DM Reg IM Reg DM Reg July 10, 2019 Stalls and flushes

Complete pipelined datapath...so far
ID/EX WB EX/MEM Control M WB MEM/WB IF/ID EX M WB PC Read register 1 Read data 1 1 2 Addr Instr Read register 2 ALU Zero ALUSrc Write register Read data 2 Result Address 1 2 Instruction memory 1 Data memory Write data Registers Write data Read data Instr [15 - 0] 1 RegDst Extend Rt 1 Rd EX/MEM.RegisterRd Rs Forwarding Unit MEM/WB.RegisterRd July 10, 2019 Stalls and flushes

What about loads? Imagine if the first instruction in the example was LW instead of SUB. How does this change the data hazard? Clock cycle DM Reg IM lw $2, 20($3) and $12, $2, $5 DM Reg IM July 10, 2019 Stalls and flushes

What about loads? Imagine if the first instruction in the example was LW instead of SUB. The load data doesn’t come from memory until the end of cycle 4. But the AND needs that value at the beginning of the same cycle! This is a “true” data hazard—the data is not available when we need it. Clock cycle DM Reg IM lw $2, 20($3) and $12, $2, $5 DM Reg IM July 10, 2019 Stalls and flushes

Stalling The easiest solution is to stall the pipeline.
We could delay the AND instruction by introducing a one-cycle delay into the pipeline, sometimes called a bubble. Notice that we’re still using forwarding in cycle 5, to get data from the MEM/WB pipeline register to the ALU. Clock cycle DM Reg IM lw $2, 20($3) and $12, $2, $5 IM Reg DM Reg July 10, 2019 Stalls and flushes

Stalling and forwarding
Without forwarding, we’d have to stall for two cycles to wait for the LW instruction’s writeback stage. In general, you can always stall to avoid hazards—but dependencies are very common in real code, and stalling often can reduce performance by a significant amount. Clock cycle DM Reg IM lw $2, 20($3) and $12, $2, $5 IM Reg DM Reg July 10, 2019 Stalls and flushes

Stalling delays the entire pipeline
If we delay the second instruction, we’ll have to delay the third one too. Why? (two reasons) Clock cycle DM Reg IM lw $2, 20($3) and $12, $2, $5 or $13, $12, $2 IM Reg DM Reg IM Reg DM Reg July 10, 2019 Stalls and flushes

Stalling delays the entire pipeline
If we delay the second instruction, we’ll have to delay the third one too. This is necessary to make forwarding work between AND and OR. It also prevents problems such as two instructions trying to write to the same register in the same cycle. Clock cycle DM Reg IM lw $2, 20($3) and $12, $2, $5 or $13, $12, $2 IM Reg DM Reg IM Reg DM Reg July 10, 2019 Stalls and flushes

Implementing stalls One way to implement a stall is to force the two instructions after LW to pause and remain in their ID and IF stages for one extra cycle. This is easily accomplished. Don’t update the PC, so the current IF stage is repeated. Don’t update the IF/ID register, so the ID stage is also repeated. Clock cycle DM Reg IM lw $2, 20($3) and $12, $2, $5 or $13, $12, $2 IM Reg Reg DM Reg IM IM Reg DM Reg July 10, 2019 Stalls and flushes

What about EXE, MEM, WB But what about the ALU during cycle 4, the data memory in cycle 5, and the register file write in cycle 6? Those units aren’t used in those cycles because of the stall, so we can set the EX, MEM and WB control signals to all 0s. Clock cycle DM Reg IM lw $2, 20($3) and $12, $2, $5 or $13, $12, $2 IM Reg Reg DM Reg IM IM Reg DM Reg July 10, 2019 Stalls and flushes

Pipelined datapath and control
Cycle 2 in Pipeline IF: and $12, $2, $5 ID: lw $2, 20($3) EX: ??? MEM: ??? WB: ??? Read address Instruction memory [31-0] Address Write data Data 1 4 Shift left 2 Add PCSrc Result Zero ALU Imm____ register 1 register 2 register data 2 data 1 Registers rd____ rt____ IF/ID ID/EX EX/MEM MEM/WB Control M WB 1004 rs___ rt___ 1008 ___ MemToReg (?) ??? rd__ RegWrite (?) MemWrite (?) MemRead (?) ALUSrc (?) ALUOp (???) RegDst (?) P C Sign extend EX July 10, 2019 Pipelined datapath and control

Cycle 3 IF: or $13, $12, $2 ID: and $12, $2, $5 EX: lw $2, 20($3) MEM: ??? WB: ??? WB MemToReg (?) Read address Instruction memory [31-0] Address Write data Data MemWrite (?) MemRead (?) 1 Shift left 2 Add PCSrc ALUSrc (___) Result Zero ALU ALUOp (___) X RegDst (___) register 1 register 2 register data 2 data 1 Registers 12 IF/ID ID/EX EX/MEM MEM/WB Control M 1008 2 5 102 105 ___ __ ??? RegWrite (?) P C Sign extend EX July 10, 2019 Pipelined datapath and control

Cycle 4 IF: or $13, $12, $2 ID: and $12, $2, $5 EX: and $12, $2, $5 MEM: lw $2, 20($3) WB: ??? Read address Instruction memory [31-0] Address Write data Data MemWrite (___) MemRead (___) 1 MemToReg (?) 4 Shift left 2 Add PCSrc ALUSrc (0) Result Zero ALU ALUOp (and) X RegDst (0) register 1 register 2 register data 2 data 1 Registers RegWrite (?) 12 IF/ID ID/EX EX/MEM MEM/WB Control M WB 1008 2 5 1012 102 105 ___ __ ??? P C Sign extend EX ___ ___ July 10, 2019 Pipelined datapath and control

Cycle 5, Data is ready July 10, 2019 Stalls and flushes ID/EX EX/MEM
WB EX/MEM Control M WB MEM/WB IF/ID EX M WB PC Read register 1 Read data 1 1 2 Addr Instr Read register 2 ALU Zero ALUSrc Write register Read data 2 Result Address 1 2 Instruction memory 1 Data memory Write data Registers Write data Read data Instr [15 - 0] 1 RegDst Extend Rt 1 Rd EX/MEM.RegisterRd Rs Forwarding Unit MEM/WB.RegisterRd July 10, 2019 Stalls and flushes

Stall = Nop conversion Clock cycle 1 2 3 4 5 6 7 8 lw $2, 20($3)
DM Reg IM lw $2, 20($3) and -> nop and $12, $2, $5 or $13, $12, $2 IM Reg DM Reg IM Reg DM Reg IM Reg DM Reg The effect of a load stall is to insert an empty or nop (“no operation”) instruction into the pipeline July 10, 2019 Stalls and flushes

Detecting stalls Detecting stall is much like detecting data hazards.
Recall the format of hazard detection equations: if (EX/MEM.RegWrite = 1 and EX/MEM.RegisterRd = ID/EX.RegisterRs) then Bypass Rs from EX/MEM stage latch DM Reg IM sub $2, $1, $3 and $12, $2, $5 mem\wb if/id id/ex ex/mem ex/mem mem\wb if/id id/ex July 10, 2019 Stalls and flushes

Detecting Stalls, cont. When should stalls be detected? lw $2, 20($3)
DM Reg IM lw $2, 20($3) and $12, $2, $5 ex/mem mem\wb if/id id/ex mem\wb IM if/id Reg if/id Reg id/ex ex/mem DM Reg What is the stall condition? if ( ID/EX.MEM_READ && // 1st is load && ((ID/EX.Rd == IF/ID.Rs) || // 1st dest is 2nd src (ID/EX.Rd == IF/ID.Rt)) ) then stall July 10, 2019 Stalls and flushes

Detecting stalls We can detect a load hazard between the current instruction in its ID stage and the previous instruction in the EX stage just like we detected data hazards. A hazard occurs if the previous instruction was LW... ID/EX.MemRead = 1 ...and the LW destination is one of the current source registers. ID/EX.RegisterRt = IF/ID.RegisterRs or ID/EX.RegisterRt = IF/ID.RegisterRt The complete test for stalling is the conjunction of these two conditions. if (ID/EX.MemRead = 1 and ( ID/EX.RegisterRt = IF/ID.RegisterRs or ID/EX.RegisterRt = IF/ID.RegisterRt)) then stall July 10, 2019 Stalls and flushes

Adding hazard detection to the CPU
Unit ID/EX WB EX/MEM M WB MEM/WB Control EX M WB PC IF/ID Read register 1 Read data 1 1 2 Addr Instr Read register 2 ALU Zero ALUSrc Write register Read data 2 Result Address 1 2 Instruction memory 1 Data memory Write data Registers Write data Read data Instr [15 - 0] 1 RegDst Extend Rt 1 Rd EX/MEM.RegisterRd Rs Forwarding Unit MEM/WB.RegisterRd July 10, 2019 Stalls and flushes

Adding hazard detection to the CPU
IF/ID Write Rs 1 Addr Instruction memory Instr Address Write data Data Read PC Extend ALUSrc Result Zero ALU Instr [15 - 0] RegDst register 1 register 2 register data 2 data 1 Registers Rd Rt IF/ID ID/EX EX/MEM MEM/WB EX M WB Control 2 Forwarding Unit EX/MEM.RegisterRd MEM/WB.RegisterRd Hazard ID/EX.MemRead PC Write ID/EX.RegisterRt July 10, 2019 Stalls and flushes

The hazard detection unit
The hazard detection unit’s inputs are as follows. IF/ID.RegisterRs and IF/ID.RegisterRt, the source registers for the current instruction. ID/EX.MemRead and ID/EX.RegisterRt, to determine if the previous instruction is LW and, if so, which register it will write to. By inspecting these values, the detection unit generates three outputs. Two new control signals PCWrite and IF/ID Write, which determine whether the pipeline stalls or continues. A mux select for a new multiplexer, which forces control signals for the current EX and future MEM/WB stages to 0 in case of a stall. July 10, 2019 Stalls and flushes

Generalizing Forwarding/Stalling
What if data memory access was so slow, we wanted to pipeline it over 2 cycles? How many bypass inputs would the muxes in EXE have? Which instructions in the following require stalling and/or bypassing? lw r13, 0(r11) add r7, r8, r9 add r15, r7, r13 Clock cycle DM Reg IM April 2, 2003 Stalls and flushes Stall Bypass

Branches in the original pipelined datapath
1 When are they resolved? ID/EX WB EX/MEM PCSrc Control M WB MEM/WB IF/ID EX M WB 4 Add Add P C Shift left 2 RegWrite Read register 1 Read data 1 MemWrite ALU Read address Instruction [31-0] Zero Read register 2 Read data 2 1 Result Address Write register Data memory Instruction memory MemToReg Registers ALUOp Write data ALUSrc Write data Read data 1 Instr [15 - 0] Sign extend RegDst MemRead Instr [ ] 1 Instr [ ] July 10, 2019 Stalls and flushes

Branches Most of the work for a branch computation is done in the EX stage. The branch target address is computed. The source registers are compared by the ALU, and the Zero flag is set or cleared accordingly. Thus, the branch decision cannot be made until the end of the EX stage. But we need to know which instruction to fetch next, in order to keep the pipeline running! This leads to what’s called a control hazard. Clock cycle beq $2, $3, Label ? ? ? IM Reg DM Reg IM July 10, 2019 Stalls and flushes

Stalling is one solution
Again, stalling is always one possible solution. Here we just stall until cycle 4, after we do make the branch decision. Clock cycle beq $2, $3, Label ? ? ? IM Reg DM Reg IM Reg DM Reg July 10, 2019 Stalls and flushes

Branch prediction Another approach is to guess whether or not the branch is taken. In terms of hardware, it’s easier to assume the branch is not taken. This way we just increment the PC and continue execution, as for normal instructions. If we’re correct, then there is no problem and the pipeline keeps going at full speed. Clock cycle beq $2, $3, Label next instruction 1 next instruction 2 IM Reg DM Reg IM Reg DM Reg IM Reg DM Reg July 10, 2019 Stalls and flushes

Branch misprediction If our guess is wrong, then we would have already started executing two instructions incorrectly. We’ll have to discard, or flush, those instructions and begin executing the right ones from the branch target address, Label. Clock cycle beq $2, $3, Label next instruction 1 next instruction 2 Label: . . . IM Reg DM Reg IM Reg flush IM flush IM Reg DM Reg July 10, 2019 Stalls and flushes

Performance gains and losses
Overall, branch prediction is worth it. Mispredicting a branch means that two clock cycles are wasted. But if our predictions are even just occasionally correct, then this is preferable to stalling and wasting two cycles for every branch. All modern CPUs use branch prediction. Accurate predictions are important for optimal performance. Most CPUs predict branches dynamically—statistics are kept at run-time to determine the likelihood of a branch being taken. The pipeline structure also has a big impact on branch prediction. A longer pipeline may require more instructions to be flushed for a misprediction, resulting in more wasted time and lower performance. We must also be careful that instructions do not modify registers or memory before they get flushed. July 10, 2019 Stalls and flushes

Implementing branches
We can actually decide the branch a little earlier, in ID instead of EX. Our sample instruction set has only a BEQ. We can add a small comparison circuit to the ID stage, after the source registers are read. Then we would only need to flush one instruction on a misprediction. Clock cycle beq $2, $3, Label next instruction 1 Label: . . . IM Reg DM Reg IM flush IM Reg DM Reg July 10, 2019 Stalls and flushes

Implementing flushes We must flush one instruction (in its IF stage) if the previous instruction is BEQ and its two source registers are equal. We can flush an instruction from the IF stage by replacing it in the IF/ID pipeline register with a harmless nop instruction. MIPS uses sll $0, $0, 0 as the nop instruction. This happens to have a binary encoding of all 0s: Flushing introduces a bubble into the pipeline, which represents the one-cycle delay in taking the branch. The IF.Flush control signal shown on the next page implements this idea, but no details are shown in the diagram. July 10, 2019 Stalls and flushes

Branching without forwarding and load stalls
1 ID/EX WB EX/MEM IF/ID Control M WB MEM/WB PCSrc EX M WB 4 The other stuff just won’t fit! P C Add Shift left 2 Read register 1 Read data 1 ALU Addr Instr Read register 2 Zero = ALUSrc Result Write register Read data 2 Address 1 Instruction memory Data memory Write data Registers Write data Read data 1 RegDst IF.Flush Extend Rt 1 Rd July 10, 2019 Stalls and flushes

Summary Three kinds of hazards conspire to make pipelining difficult.
Structural hazards result from not having enough hardware available to execute multiple instructions simultaneously. These are avoided by adding more functional units (e.g., more adders or memories) or by redesigning the pipeline stages. Data hazards can occur when instructions need to access registers that haven’t been updated yet. Hazards from R-type instructions can be avoided with forwarding. Loads can result in a “true” hazard, which must stall the pipeline. Control hazards arise when the CPU cannot determine which instruction to fetch next. We can minimize delays by doing branch tests earlier in the pipeline. We can also take a chance and predict the branch direction, to make the most of a bad situation. July 10, 2019 Stalls and flushes

Stalls and flushes Last time, we discussed data hazards that can occur in pipelined CPUs if some instructions depend upon others that are still executing.

Similar presentations

Presentation on theme: "Stalls and flushes Last time, we discussed data hazards that can occur in pipelined CPUs if some instructions depend upon others that are still executing."— Presentation transcript:

Similar presentations

About project

Feedback

Log in

Auth with social network:

Stalls and flushes Last time, we discussed data hazards that can occur in pipelined CPUs if some instructions depend upon others that are still executing.

Similar presentations

Presentation on theme: "Stalls and flushes Last time, we discussed data hazards that can occur in pipelined CPUs if some instructions depend upon others that are still executing."— Presentation transcript:

Similar presentations

About project

Feedback