1. Computer Architecture: A Constructive Approach Branch Direction Prediction – Six Stage Pipeline Joel Emer Computer Science & Artificial Intelligence Lab. Massachusetts Institute of Technology April 23, 2012L19-1 http://csg.csail.mit.edu/6.S078

2. NA pred with decode feedback April 23, 2012 L19-2http://csg.csail.mit.edu/6.S078 F Fetch D Decode R Reg Read X Execute M Memory W Write- back Next Address Prediction

3. Decode detected mispredicts Non-branch When nextPC != PC+4 => use PC+4 Unconditional target known at decode When nextPC != known target => use known target Conditional branch When nextPC != PC+4 or decoded target => use PC+4 April 23, 2012 L19-3http://csg.csail.mit.edu/6.S078 Can we do better than PC+4?

4. Dynamic Branch Prediction April 23, 2012 L12-4http://csg.csail.mit.edu/6.S078 Branch direction prediction: Learn and predict the direction a branch will go Standard prediction principles: Temporal correlation The way a branch resolves may be a good predictor of the way it will resolve at the next execution Spatial correlation Several branches may resolve in a highly correlated manner (a preferred path of execution)

5. One-bit predictor April 23, 2012 L19-5http://csg.csail.mit.edu/6.S078 00 Fetch PC Branch? Target PC + I-Cache Opcodeoffset Instruction k BHT Index 2 k -entry BHT, 1 bits/entry Taken/¬Taken? Fetch Decode Predict branch will go same direction it went last time

6. One-bit predictor // Interface interface DirectionPred; method ActionValue#(Tuple2#(Bool, DirInfo)) predict(Addr addr); method Action train(DirInfo dirInfo, Bool taken); endinterface // Feedback information typedef 64 BPRows; typedef Bit#(TLog#(BPRows)) DirLineIndex; typedef DirLineIndex DirInfo; April 23, 2012 L19-6http://csg.csail.mit.edu/6.S078

7. One-bit predictor (continued) module mkDirectionPredictor(DirectionPred); RegFile#(DirLineIndex, Bool) dirArray <- mkRegFileFull(); method ActionValue#(Tuple2#(Bool, DirInfo)) predict(Addr addr); DirLineIndex index = truncate(addr >> 2); return tuple2(dirArray.sub(index), index); endmethod method Action train(DirInfo dirInfo, Bool taken); DirLineIndex index = dirInfo; dirArray.upd(index, taken); endmethod endmodule April 23, 2012 L19-7http://csg.csail.mit.edu/6.S078  Array of prediction bits  Return prediction saved in array  Update array with last actual behavior When should we train?

8. Two-bit Predictor Smith, 1981 April 23, 2012 L19-8http://csg.csail.mit.edu/6.S078 Assume 2 direction prediction bits per instruction On ¬taken   On taken 11Strongly taken 10Weakly taken 01Weakly ¬taken 00Strongly ¬taken How well does one-bit predictor do on short trip count loops? Implement using saturating counter

9. Saturating Counter typedef Bit#(2) Counter; function Counter updateCounter(Bool dir, Counter counter); return dir?saturatingInc(counter) :saturatingDec(counter); endfunction function Counter saturatingInc(Counter counter); let plusOne = counter + 1; return (plusOne == 0)?counter:plusOne; endfunction function Counter saturatingDec(Counter counter); return (counter == 0)?0:counter-1; endfunction April 23, 2012 L19-9http://csg.csail.mit.edu/6.S078 How do we determine prediction from counter?

10. Two-bit predictor April 23, 2012 L19-10http://csg.csail.mit.edu/6.S078 00 Fetch PC k BHT Index 2 k -entry BHT, 1 bits/entry Taken/¬Taken?

11. Two-bit predictor typedef 64 BPRows; typedef Bit#(TLog#(BPRows)) DirLineIndex; // DirInfo data typedef struct { DirLineIndex index; Counter counter; } DirInfo deriving(Bits, Eq); module mkDirectionPredictor(DirectionPred); // Direction predictor state RegFile#(DirLineIndex,Counter) cntArray <- mkRegFileFull(); April 23, 2012 L19-11http://csg.csail.mit.edu/6.S078  Feedback state for training

12. Two-bit predictor (continued) method ActionValue#(Tuple2#(Bool, DirInfo)) predict(Addr addr); DirInfo info = ? info.index = truncate(addr >> 2); info.counter = cntArray.sub(index); Bool taken = (truncate(counter >> 1) == 1); return tuple2(taken, info); endmethod method Action train(DirInfo info, Bool taken); cntArray.upd(info.index, updateCounter(taken, info.counter)); endmethod endmodule April 23, 2012 L19-12http://csg.csail.mit.edu/6.S078  Training information is index and counter  Prediction is high bit of counter  Train by updating counter

13. Exploiting Spatial Correlation Yeh and Patt, 1992 April 23, 2012 L19-13http://csg.csail.mit.edu/6.S078 Implemented with a history register, ‘hist’, that records the direction of the last N branches executed by the processor. if (x[i] < 7) then y += 1; if (x[i] < 5) then c -= 4; If first condition false, second condition also false Also works well for short trip count loops.

14. Ghist predictor typedef 64 BPRows; typedef Bit#(TLog#(BPRows)) DirLineIndex; typedef Bit#(2) Counter; // DirInfo data typedef struct { DirLineIndex hist; Counter counter; } DirInfo deriving(Bits, Eq); module mkDirectionPredictor(DirectionPred); // Direction predictor state Reg#(DirLineIndex) hist <- mkReg(0); RegFile#(DirLineIndex,Counter) cntArray <- mkRegFileFull(); April 23, 2012 L19-14http://csg.csail.mit.edu/6.S078

15. Global history predictor method ActionValue#(Tuple2#(Bool, DirInfo)) predict(Addr addr); DirInfo info = ?; info.hist = hist; info.counter = cntArray.sub(hist); Bit#(1) pred = truncate(info.counter >> 1); hist <= truncate(hist << 1 | zeroExtend(pred)); return tuple2((pred == 1), info); endmethod April 23, 2012 L19-15http://csg.csail.mit.edu/6.S078 How good are predictions while waiting for training?  Shift new prediction into history register  Calculate feedback information

16. Global history predictor method Action train(DirInfo info, Bool taken); counterArray.upd(info.hist, updateCounter(taken, info.counter)); endmethod method Action repair(DirInfo info, Bool taken); hist <= truncate((info.hist << 1) | zeroExtend(pack(taken))); endmethod endmodule April 23, 2012 L19-16http://csg.csail.mit.edu/6.S078 What is the state of ‘hist’ after redirects from decode and execute?  Restore history to state it would be in after the desired prediction

17. NA pred with decode feedback April 23, 2012 L19-17http://csg.csail.mit.edu/6.S078 F Fetch D Decode R Reg Read X Execute M Memory W Write- back Next Address Prediction Direction Prediction

18. Direction prediction recipe Execute Send redirects on mispredicts (unchanged) Send direction prediction training Decode Check if next address matches direction pred Send redirect if different Fetch Generate prediction Learn from feedback Accept redirects from later stages April 23, 2012 L19-18http://csg.csail.mit.edu/6.S078

19. Add direction feedback typedef struct { Bool correct; NaInfo naPredInfo; Addr nextAddr; DirInfo dirPredInfo; Bool taken; } Feedback deriving (Bits, Eq); FIFOF#(Tuple3#(Epoch,Epoch,Feedback)) decFeedback<-mkFIFOF; FIFOF#(Tuple2#(Epoch,Feedback)) execFeedback <- mkFIFOF; April 23, 2012 L19-19http://csg.csail.mit.edu/6.S078  Feedback needs information for training direction predictor

20. Execute (branch analysis) // after executing instruction... let nextEeEpoch = eeEpoch; let cond = execData.execInst.cond; let nextPc = cond?execData.execInst.addr : execData.pc+4; if (nextPC != execData.nextAddrPred) nextEeEpoch += 1; eeEpoch <= newEeEpoch; execFeedback.enq(tuple2(nextEeEpoch, Feedback{correct: (nextPC == execData.nextAddrPred), taken: cond, dirPredInfo: execData.dirPredInfo, naPredInfo: execData.naPredInfo, nextAddr: nextPc})); // enqueue instruction to next stage April 23, 2012 L19-20http://csg.csail.mit.edu/6.S078  Recall: may have been set in decode  Always send feedback

21. Decode with mispredict detect rule doDecode; let decData = newDecData(fr.first); let correctPath = (decData.execEpoch != deEpoch) ||(decData.decEpoch == ddEpoch); let instResp = decData.fInst.instResp; let pcPlus4 = decData.pc+4; if (correctPath) begin decData.decInst = decode(instResp, pcPlus4); let target = knownTargetAddr(decData.decInst); let brClass = getBrClass(decData.decInst); let predTarget = decData.nextAddrPred; let predDir = decData.takenPred; April 23, 2012 L19-21http://csg.csail.mit.edu/6.S078  Determine if epoch of incoming instruction is on good path  New exec epoch  Same dec epoch

22. Decode with mispredict detect let decodedTarget = case (brClass) NonBranch: pcPlus4; UncondKnown: target; CondBranch: (predDir?target:pcPlus4); default: decData.nextAddrPred; endcase; if (decodedTarget != predTarget) begin decData.decEpoch = decData.decEpoch + 1; decData.nextAddrPred = decodedTarget; decFeedback.enq( tuple3(decData.execEpoch, decData.decEpoch, Feedback{correct: False, naPredInfo: decData.naPredInfo, nextAddr: decodedTarget, dirPredInfo: decData.dirPredInfo, taken: decData.takenPred})); end dr.enq(decData); end // of correct path April 23, 2012 L19-22http://csg.csail.mit.edu/6.S078  Wrong next addr?  Tell exec addr of next instruction!  Send feedback  New dec epoch  Enqueue to next stage on correct path  Calculate target as best as decode can

23. Decode with mispredict detect else begin // incorrect path decData.decEpoch = ddEpoch; decData.execEpoch = deEpoch; end ddEpoch <= decData.decEpoch; deEpoch <= decData.execEpoch; fr.deq; endrule April 23, 2012 L19-23http://csg.csail.mit.edu/6.S078  Preserve current epoch if instruction on incorrect path decData.*Epoch have been set properly so we always save them.

24. Handling redirect from execute if (execFeedback.notEmpty) begin match {.execEpoch,.fb} = execFeedback.first; execFeedback.deq; if(!fb.correct) begin dirPred.repair(fb.dirPredInfo, fb.taken); dirPred.train(fb.dirPredInfo, fb.taken); naPred.repair(fb.naPredInfo, fb.nextAddr); naPred.train(fb.naPredInfo, fb.nextAddr); feEpoch <= execEpoch; fetchPc <= feedback.nextAddr; end else begin dirPred.train(fb.dirPredInfo, fb.taken); naPred.train(fb.naPredInfo, fb.nextAddr); enqInst; end April 23, 2012 L19-24http://csg.csail.mit.edu/6.S078 Train and repair on redirect Just train on correct prediction

25. Handling redirect from decode else if (decFeedback.notEmpty) begin decFeedback.deq; match {.execEpoch,.decEpoch,.fb} = decFeedback.first; if (execEpoch == feEpoch) begin if (!fb.correct) begin // epoch unchanged fdEpoch <= decEpoch; dirPred.repair(fb.dirPredInfo, fb.taken); naPred.repair(fb.naPredInfo, fb.nextAddr); fetchPc <= feedback.nextAddr; end else // dec feedback on correct prediction enqInst; end else // dec feedback, but in fetch is in new exec epoch enqInst; else // no feedback enqInst; April 23, 2012 L19-25http://csg.csail.mit.edu/6.S078 Just repair never train on feedback from decode