1. Meta Optimization Improving Compiler Heuristics with Machine Learning Mark Stephenson, Una-May O’Reilly, Martin Martin, and Saman Amarasinghe MIT Computer Architecture Group Presented by Utku Aydonat

2. Motivation Compiler writers are faced with many challenges: –Many compiler problems are NP-hard –Modern architectures are inextricably complex Simple models can’t capture architecture intricacies –Micro-architectures change quickly

3. Motivation Heuristics alleviate complexity woes –Find good approximate solutions for a large class of applications –Find solutions quickly Unfortunately… –They require a lot of trial-and-error tweaking to achieve suitable performance –Fine tuning is a tedious process

4. Priority Functions A single priority or cost function often dictates the efficacy of a heuristic Priority Function: A function of the factors that affect a given problem Priority functions rank the options available to a compiler heuristic

5. Priority Functions Graph coloring register allocation (selecting nodes to spill) List scheduling (identifying instructions in worklist to schedule first) Hyperblock formation (selecting paths to include) Data Prefetching (inserting prefetch instructions)

6. List Scheduling

7. Machine Learning They propose using machine learning techniques to automatically search the priority function space –Feedback directed optimization –Find a function that works well for a broad range of applications –Needs to be applied only once

8. Generic Programming Modeled after Darwinism (Survival of the Fittest). Parse trees of operators and operands describe the potential priority functions. A population is a collection of parse trees for one generation. After testing, several members of the population are selected for reproduction via crossover, which swaps a random node from each of 2 parse trees. Other parse trees are selected for mutation, in which a random node is replaced by a random expression.

9. Generic Programming

10. Genetic Programming Create initial population (initial solutions) Evaluation Selection END Generation of variants (mutation and crossover) Generations < Limit?

11. Generic Programming The system selects the smaller of several expressions that are equally fit so that the parse trees do not grow exponentially. Tournament selection is used repeatedly to select parse trees for crossover. –Choose N expressions at random from the population and select the one with the highest fitness. Dynamic subset selection (DSS) is used to reduce fitness evaluations to achieve suitable solution.

12. Meta Optimization Just as with Natural Selection, the fittest individuals are more likely to survive and reproduce. Expressions creating fastest code are the fittest. Create 399 random expressions based on parameters. It also seeded with the compiler writer’s best guesses

13. Meta Optimization

14. Find predictable regions of control flow Prioritize paths based on several characteristics –The priority function we want to optimize Add paths to hyperblock in priority order Case Study I: Hyperblock Formation

15. Hyperblock Formation

16. Case Study I: IMPACT’s Function Favor frequently Executed paths Favor short paths Favor parallel paths Penalize paths with hazards

17. Hyperblock Formation What are the important characteristic of a hyperblock formation priority function? IMPACT uses four characteristics Extract all the characteristics you can think of and have a machine learning algorithm find the priority function

18. Hyperblock Formation x1 Maximum ops over pathsx2 Dependence height x3 Number of pathsx4 Number of operations x5 Does path have subroutine calls?x6 Number of branches x7 Does path have unsafe calls?x8 Path execution ratio x9 Does path have pointer derefs?x10 Average ops executed in path x11 Issue width of processorx12 Average predictability of branches in path …xN Predictability product of branches in path

19. Hyperblock Results Compiler Specialization 1.54 1.23 0 0.5 1 1.5 2 2.5 3 3.5 129.compress g721encodeg721decode huff_dec huff_enc rawcaudiorawdaudio toast mpeg2dec Average Speedup Train data setAlternate data set

20. Hyperblock Results A General Purpose Priority Function

21. Cross Validation Testing General Purpose Applicability

22. Case Study II: Register Allocation A General Purpose Priority Function

23. Register Allocation Results Cross Validation

24. Case Study III: Prefetching A General Purpose Priority Function

25. Prefecthing Results Cross Validation

26. Conclusion Machine learning techniques can identify effective priority functions ‘Proof of concept’ by evolving three well known priority functions Human cycles v. computer cycles

27. My Conclusions Heuristics to improve heuristics –How to choose population size, mutation rate, tournament size? Does it guarantee better results? Requires a lot of experiments for the results to converge. –Do we have that opportunity? It is very effective for optimizing a specific application space.

28. Thank you!

29. (add (sub (mul exec_ratio_mean 0.8720) 0.9400) (mul 0.4762 (cmul (not has_pointer_deref) (mul 0.6727 num_paths) (mul 1.1609 (add (sub (mul (div num_ops dependence_height) 10.8240) exec_ratio) (sub (mul (cmul has_unsafe_jsr predict_product_mean 0.9838) (sub 1.1039 num_ops_max)) (sub (mul dependence_height_mean num_branches_max) num_paths))))))) GP Hyperblock Solutions General Purpose Intron that doesn’t affect solution

30. (add (sub (mul exec_ratio_mean 0.8720) 0.9400) (mul 0.4762 (cmul (not has_pointer_deref) (mul 0.6727 num_paths) (mul 1.1609 (add (sub (mul (div num_ops dependence_height) 10.8240) exec_ratio) (sub (mul (cmul has_unsafe_jsr predict_product_mean 0.9838) (sub 1.1039 num_ops_max)) (sub (mul dependence_height_mean num_branches_max) num_paths))))))) GP Hyperblock Solutions General Purpose Favor paths that don’t have pointer dereferences

31. GP Hyperblock Solutions General Purpose (add (sub (mul exec_ratio_mean 0.8720) 0.9400) (mul 0.4762 (cmul (not has_pointer_deref) (mul 0.6727 num_paths) (mul 1.1609 (add (sub (mul (div num_ops dependence_height) 10.8240) exec_ratio) (sub (mul (cmul has_unsafe_jsr predict_product_mean 0.9838) (sub 1.1039 num_ops_max)) (sub (mul dependence_height_mean num_branches_max) num_paths))))))) Favor highly parallel (fat) paths

32. GP Hyperblock Solutions General Purpose (add (sub (mul exec_ratio_mean 0.8720) 0.9400) (mul 0.4762 (cmul (not has_pointer_deref) (mul 0.6727 num_paths) (mul 1.1609 (add (sub (mul (div num_ops dependence_height) 10.8240) exec_ratio) (sub (mul (cmul has_unsafe_jsr predict_product_mean 0.9838) (sub 1.1039 num_ops_max)) (sub (mul dependence_height_mean num_branches_max) num_paths))))))) If a path calls a subroutine that may have side effects, penalize it

33. Case Study I: IMPACT’s Algorithm A B C E F G 4k24k 4k22k2k 25 28k D 10 Pathexechazopsdeppr A-B-D-F-G~01.0134~0 A-B-F-G0.141.01040.21 A-C-F-G0.791.0921.44 A-C-E-F-G0.070.251350.02 A-C-E-G~00.25113~0

34. Case Study I: IMPACT’s Algorithm A B C E F G 4k24k 4k22k2k 25 28k D 10 Pathexechazopsdeppr A-B-D-F-G~01.0134~0 A-B-F-G0.141.01040.21 A-C-F-G0.791.0921.44 A-C-E-F-G0.070.251350.02 A-C-E-G~00.25113~0