1. Wish Branches A Review of “Wish Branches: Enabling Adaptive and Aggressive Predicated Execution” Russell Dodd - October 24, 2006

2. Motivation Traditional Execution Control dependencies can cause delays or stalls. Performance penalties for hard to predict branches. Predicated Execution Converts control dependencies in branches into data dependencies. Typical HW cannot override a compiler’s choice of branch type. If easy-to-predict branches are predicated (unnecessarily); it results in processing overhead and possibly a performance loss. Cannot remove backward branches. Russell Dodd - October 24, 2006 Adapted from H. Kim et al

3. Proposed Solution Compiler generates code that has dual functionality: predicated and non-predicated. Same as predicated code only predicated conditional branches are not removed. These conditional branches are called wish branches. Russell Dodd - October 24, 2006 Adapted from H. Kim et al

4. Proposed Solution Wish Branches When fetched, HW estimates the difficulty of the branch prediction at runtime with a confidence estimator. If hard-to-predict the HW executes the predicated version of the code, to avoid misprediction. Otherwise, uses branch predictor and ignores predicate information for easy-to-predict branches. Russell Dodd - October 24, 2006 Adapted from H. Kim et al

5. Comparison Figure 1 from “Wish Branches: Enabling Adaptive and Aggressive Predicated Execution” Russell Dodd - October 24, 2006 Adapted from H. Kim et al C-code

6. Comparison Figure 1 from “Wish Branches: Enabling Adaptive and Aggressive Predicated Execution” Russell Dodd - October 24, 2006 Adapted from H. Kim et al C-code Condition==FALSE Low confidence mode Not used High confidence mode

7. Consequences If the confidence estimate is accurate Avoids overhead of predicated execution. Or eliminates a branch misprediction. If the wish branch is mispredicted In high confidence the pipeline must be flushed. In low confidence the processor does not flush the pipeline, but executes as conventional predicated code. Russell Dodd - October 24, 2006 Adapted from H. Kim et al

8. Wish Loops Used for backward branching. Used since predication cannot directly remove backward branches. Use predication to reduce branch misprediction penalties, without removing the branch. Russell Dodd - October 24, 2006 Adapted from H. Kim et al

9. Comparison Extra instruction to initialize predicate C-code Russell Dodd - October 24, 2006 Adapted from H. Kim et al Figure 2 from “Wish Branches: Enabling Adaptive and Aggressive Predicated Execution”

10. A Closer Look at the Wish Loop in Low Confidence C-Code do{ a++; i++; } while (i<3) a = 1 i = 1 i < 3 a= 2 i = 2 i < 3 a = 3 i = 3 i !< 3 exit What if a misprediction occurs during the loop? Three cases can occur: 1 Early Exit The loop iterates fewer times than it should. Processor needs to execute the body one more time. Requires a pipeline flush just like a normal mispredicted branch. Early stop here Russell Dodd - October 24, 2006 Adapted from H. Kim et al

11. A Closer Look at the Wish Loop in Low Confidence C-Code do{ a++; i++; } while (i<3) a = 1 i = 1 i < 3 a= 2 i = 2 i < 3 a = 3 i = 3 i !< 3 exit a = 4 i = 4 i !< 3 exit What if a misprediction occurs during the loop? Three cases can occur: 2 Late Exit The loop iterates a few more times than it should. Allow extra blocks to go through data path as no-ops. Performs better than normal backward branch since it reduces penalty. Late stop here Russell Dodd - October 24, 2006 Adapted from H. Kim et al

12. A Closer Look at the Wish Loop in Low Confidence C-Code do{ a++; i++; } while (i<3) a = 1 i = 1 i < 3 a= 2 i = 2 i < 3 a = 3 i = 3 i !< 3 exit … a = N i = N i !< 3 exit What if a misprediction occurs during the loop? Three cases can occur: 3 No Exit Flush any extra blocks and continue. Similar to normal mispredicted branch. If extra blocks were treated as no-ops as in late exit it would waste energy. No exit Russell Dodd - October 24, 2006 Adapted from H. Kim et al

13. Wish Branches in Complex Control Flow NormalPredicatedWish Branch C-code Russell Dodd - October 24, 2006 Adapted from H. Kim et al Figure 3 from “Wish Branches: Enabling Adaptive and Aggressive Predicated Execution”

14. Wish Branch Support ISA Implementable in existing branch instructions. Requires two hint bits for four choices (normal branch, wish jump, wish join and wish loop). Compiler Needs to decide which branches should be converted to. Hardware Requires an accurate confidence estimator. Changes to the processor front-end and branch misprediction detection module. Russell Dodd - October 24, 2006 Adapted from H. Kim et al

15. Advantages Dynamically eliminates the performance overhead of predicated execution. Allows compilers to generate code more aggressively, since the processor corrects the decisions at runtime. Exploits predicated execution to reduce branch misprediction penalties for backward branches. Increases code adaptivity to machines since processors dynamically decide when not to use predication. Russell Dodd - October 24, 2006 Adapted from H. Kim et al

16. Disadvantages Requires extra branch instructions. Increases contention for the branch predictor table entries. Increase interference in pattern history tables. Reduces the size of basic code blocks by adding control dependencies. Reduces the scope of optimization. Russell Dodd - October 24, 2006 Adapted from H. Kim et al

17. Performance Evaluation Implemented in the IA-64 ISA since it has full support for predication. Used a superscalar out-of-order processor model. Russell Dodd - October 24, 2006 Adapted from H. Kim et al An excerpt from “Wish Branches: Enabling Adaptive and Aggressive Predicated Execution” pg 54.

18. Results Figure 4 from “Wish Branches: Enabling Adaptive and Aggressive Predicated Execution” Russell Dodd - October 24, 2006 Adapted from H. Kim et al Creates large skew on average.

19. Results Russell Dodd - October 24, 2006 Adapted from H. Kim et al Table 1 from “Wish Branches: Enabling Adaptive and Aggressive Predicated Execution”

20. Analysis Wish branches improve performance since they divide work between the compiler and micro-architecture. Compilers excel at analyzing control-flow graphs and making predicated code. Micro-architectures excel at making runtime decisions: whether to use predicated execution or branch prediction, based on dynamic program information. Adapted from H. Kim et al Russell Dodd - October 24, 2006

21. Future Directions Develop computer algorithms to decide which branches to convert to wish branches. Combat reduced scope for code optimization. Develop more accurate confidence estimation mechanisms. Develop a hardware wish loop predictor. Russell Dodd - October 24, 2006 Adapted from H. Kim et al

22. My Thoughts Branches are large portion of code content ~ 20%, so it’s worth the effort to improve. No mention of code size/instruction count. Dynamic micro-instruction count given in previous paper but no comparison given to other styles. Branch Predicting vs. Wish Branching Hypothetically… If a branch predictor can be improved, so it is accurate most of the time during runtime, is it worthwhile to invest in dynamically solving branch-types (predicated/normal) for further prediction. Russell Dodd - October 24, 2006

23. My Thoughts More hardware isn’t that big of a deal in a billion transistor world. (A previous paper details hardware requirements and its complexity) H.Kim et al., “Wish Branches: Combining Conditional Branching and Predication for Adaptive Predicated Execution”. Software would not require a large investment since it is based on predicated (existing) execution. However compiler support to enable heuristics for optimization could be expensive. How much?? The improvement in execution time is a really interesting development for future processors. How far could prediction be nested? Predict the confidence estimator’s results.. Russell Dodd - October 24, 2006

24. END