1. Evaluation of the Gini-index for Studying Branch Prediction Features Veerle Desmet Lieven Eeckhout Koen De Bosschere

2. 2 A simple prediction example outlook t° windy season,... umbrella prediction mechanism features goal = prediction accuracy of 100% past observations

3. 3 A simple prediction example Daily prediction Binary prediction: yes or no Outcome in the evening Prediction strategies: –No need in summer, yes otherwise Easy, not very accurate –Based on humidity and temperature More complex, very accurate

4. 4 Predicting How to improve prediction accuracy? Shortcomings of existing models? –Feature set –Prediction mechanism –Implementation limits –... This talk: evaluation of prediction features for branch prediction

5. 5 Program execution Phases during instruction execution: Fetch = read next instruction Decode = analyze type and read operands Execute Write Back = write result FetchDecodeExecuteWrite Back R1=R2+R3 addition 43 computation R1 contains 7

6. 6 Pipelined architectures Parallel versus sequential: Constant flow of instructions possible Faster applications Limitation due to branches FetchDecodeExecuteWrite Back R1=R2+R3 R5=R2+1R1=R2+R3R5=R2+1R4=R3-1R1=R2+R3R5=R2+1R4=R3-1R7=2*R1R5=R2+1R4=R3-1R7=2*R1R5=R6R4=R3-1R7=2*R1R5=R6R1=4

7. 7 Branches FetchDecodeExecuteWrite Back R1=R2+R3 R5=R2+1 R4=R3-1R7=2*R1R5=R6 test R1=0 R5=R2+1R5=R6 ? test R1=0R5=R2+1 ?? test R1=0 R7=2*R1 R1=R2+R3 R5=R2+1 R7=0 R7=2*R1 R5=R6 test R1=0 yes no R2=R2-1 Branches determine program flow or execution path Introduce 2 bubbles affecting pipeline throughput

8. 8 Solution 1 out of 8 instructions is a branch Waiting for the outcome of branches seriously affects amount of parallelism Increasing number of pipeline stages –Pentium 4: up to 20 stages Predict outcome of branch

9. 9 Branch prediction Fetch those instructions that are likely to be executed Correct prediction eliminates bubbles FetchDecodeExecuteWrite Back R1=R2+R3 R5=R2+1 R4=R3-1R7=2*R1R5=R6 test R1=0 R5=R2+1R5=R6 R7=2*R1 test R1=0R5=R2+1 R7=2*R1R2=R2-1 R1=R2+R3 R5=R2+1 R7=0 R7=2*R1 R5=R6 test R1=0 yes no R2=R2-1

10. 10 Branch prediction Prediction for each branch execution Binary prediction: taken or not-taken Outcome after the test is excuted Prediction strategies: –Many predictors in literature –Static versus dynamic

11. 11 Static branch prediction BTFNT: Backward Taken, Forward Not Taken –Loops (e.g. For, while) –Summer no need of umbrella Based on type of test in branch –Branch if equal mostly not-taken –Sunday no need of umbrella Easy, prediction fixed at compile-time Prediction accuracy: about 75%

12. 12 Dynamic branch prediction Bimodal Global Gshare Local Simulations: SimpleScalar/Alpha SPEC2000 integer benchmarks 250M branches

13. 13 Bimodal branch predictor Averaging outcomes from previous years branch address saturating counter e.g. 2 saturating counter e.g. 3 prediction e.g. taken update with outcome e.g. taken

14. 14 Global branch predictor Averaging last day outcomes global history e.g. 0111 saturating counter e.g. 2 saturating counter e.g. 3 prediction e.g. taken update with outcome e.g. taken global history e.g. 1111

15. 15 Gshare branch predictor saturating counter e.g. 2 prediction e.g. taken branch address XOR global history e.g. 1010 update with outcome AMD K6

16. 16 Local branch predictor Record day outcomes of previous years Averaging over same day histories branch address prediction local history e.g. 1111 saturating counter e.g. 2

17. 17 Accuracy versus storage local bimodal gshare global 75 80 85 90 95 100 110100100010000100000 Prediction Accuracy (%) Predictor Size (byte)

18. 18 Branch prediction strategies All saturating counter mechanism All use of limited tables –problem with so-called aliasing Different prediction features Accuracies up to 95% Further improvement? Predictive power of features?

19. 19 Feature selection Which features are relevant? Less features –require less storage –faster prediction prediction mechanism features ? Feature selection

20. 20 Systematic feature evaluation Feature = input to predictor Power of features –predictor size not fixed –prediction strategy not fixed Decision trees: –Selects feature –Split observations –Recursive algorithm –Easy understandable

21. 21 Decision Tree Construction Outlookt°windy sunnyhighno sunnylowyes overcasthighno overcastlowno overcasthighyes overcastlowyes rainlownoyes rainhighyes past observations features outlook windy YES NOYES yesno sunny overcast rain prediction mechanism prediction

22. 22 Gini-index Metric for partition purity of a data set S: Gini (S) = 1 – p 0 ² – p 1 ² where p i is the relative frequence of class i in S For binary prediction: minimum 0 maximum 0.5 The higher the Gini-index, the more difficult to predict

23. 23 Finding good split points If data set S is split into two subsets S0 and S1 with sizes N0 and N1 (N = N0 + N1): Feature with lowest Gini split is chosen Extensible for non binary features Looking for features with low Gini split -index, i.e. features with good predictive power N0 N N1 N Gini(S0) +Gini(S1) Gini split (S) =

24. 24 Individual feature bits 0 0,05 0,1 0,15 0,2 0,25 0,3 0,35 0,4 0,45 0,5 global history local history branch address gshare-index target direction branch type ending type successor basic block dynamic features static features Gini split -index

25. 25 Individual features Local history bits very good –perfect local history uses branch address Static features powerful –non-binary –except target direction –known at compile-time Looking for good feature combinations...

26. 26 Features as used in predictors 0 0,05 0,1 0,15 0,2 0,25 0,3 0,35 0,4 0,45 0,5 01234567891011121314151617181920 Gini split -index global history branch address gshare-index local history Feature length (bit)

27. 27 Features as used in predictors Static features better for small lengths Better if longer features A few local history bits enough Same behaviour as accuracy curves –low Gini-index implies high accuracy Independent to predictor size Independent to prediction strategy

28. 28 Remark Limitation of decision trees: outliers –majority vote –clean data Keep implementation in mind Outlookt°windy sunnyhighno sunnyhighnoyes sunnyhighno

29. 29 Conclusion Need of accurate branch prediction in modern microprocessors Towards systematic predictor development –Selecting features –Predictive power of features Gini-index useful for studying branch prediction features –without fixing any predictor aspect

30. Thanks for Listening

31. 31 Branch prediction Incorrect prediction or misprediction Penalty for cancelling instructions (3 bubbles) FetchDecodeExecuteWrite Back test R1=0R5=R2+1R5=R6 R7=2*R1 test R1=0R5=R2+1 R7=2*R1R2=R2-1 R1=R2+R3 R5=R2+1 R7=0 R7=2*R1 R5=R6 test R1=0 yes no R2=R2-1 R7=0 test R1=0R7=2*R1 R2=R2-1