Meta Optimization Improving Compiler Heuristics with Machine Learning Mark Stephenson, Una-May O’Reilly, Martin Martin, and Saman Amarasinghe MIT Computer Architecture Group Presented by Utku Aydonat

Motivation Compiler writers are faced with many challenges: –Many compiler problems are NP-hard –Modern architectures are inextricably complex Simple models can’t capture architecture intricacies –Micro-architectures change quickly

Motivation Heuristics alleviate complexity woes –Find good approximate solutions for a large class of applications –Find solutions quickly Unfortunately… –They require a lot of trial-and-error tweaking to achieve suitable performance –Fine tuning is a tedious process

Priority Functions A single priority or cost function often dictates the efficacy of a heuristic Priority Function: A function of the factors that affect a given problem Priority functions rank the options available to a compiler heuristic

Priority Functions Graph coloring register allocation (selecting nodes to spill) List scheduling (identifying instructions in worklist to schedule first) Hyperblock formation (selecting paths to include) Data Prefetching (inserting prefetch instructions)

List Scheduling

Machine Learning They propose using machine learning techniques to automatically search the priority function space –Feedback directed optimization –Find a function that works well for a broad range of applications –Needs to be applied only once

Generic Programming Modeled after Darwinism (Survival of the Fittest). Parse trees of operators and operands describe the potential priority functions. A population is a collection of parse trees for one generation. After testing, several members of the population are selected for reproduction via crossover, which swaps a random node from each of 2 parse trees. Other parse trees are selected for mutation, in which a random node is replaced by a random expression.

Generic Programming

Genetic Programming Create initial population (initial solutions) Evaluation Selection END Generation of variants (mutation and crossover) Generations < Limit?

Generic Programming The system selects the smaller of several expressions that are equally fit so that the parse trees do not grow exponentially. Tournament selection is used repeatedly to select parse trees for crossover. –Choose N expressions at random from the population and select the one with the highest fitness. Dynamic subset selection (DSS) is used to reduce fitness evaluations to achieve suitable solution.

Meta Optimization Just as with Natural Selection, the fittest individuals are more likely to survive and reproduce. Expressions creating fastest code are the fittest. Create 399 random expressions based on parameters. It also seeded with the compiler writer’s best guesses

Meta Optimization

Find predictable regions of control flow Prioritize paths based on several characteristics –The priority function we want to optimize Add paths to hyperblock in priority order Case Study I: Hyperblock Formation

Hyperblock Formation

Case Study I: IMPACT’s Function Favor frequently Executed paths Favor short paths Favor parallel paths Penalize paths with hazards

Hyperblock Formation What are the important characteristic of a hyperblock formation priority function? IMPACT uses four characteristics Extract all the characteristics you can think of and have a machine learning algorithm find the priority function

Hyperblock Formation x1 Maximum ops over pathsx2 Dependence height x3 Number of pathsx4 Number of operations x5 Does path have subroutine calls?x6 Number of branches x7 Does path have unsafe calls?x8 Path execution ratio x9 Does path have pointer derefs?x10 Average ops executed in path x11 Issue width of processorx12 Average predictability of branches in path …xN Predictability product of branches in path

Hyperblock Results Compiler Specialization compress g721encodeg721decode huff_dec huff_enc rawcaudiorawdaudio toast mpeg2dec Average Speedup Train data setAlternate data set

Hyperblock Results A General Purpose Priority Function

Cross Validation Testing General Purpose Applicability

Case Study II: Register Allocation A General Purpose Priority Function

Register Allocation Results Cross Validation

Case Study III: Prefetching A General Purpose Priority Function

Prefecthing Results Cross Validation

Conclusion Machine learning techniques can identify effective priority functions ‘Proof of concept’ by evolving three well known priority functions Human cycles v. computer cycles

My Conclusions Heuristics to improve heuristics –How to choose population size, mutation rate, tournament size? Does it guarantee better results? Requires a lot of experiments for the results to converge. –Do we have that opportunity? It is very effective for optimizing a specific application space.

Thank you!

(add (sub (mul exec_ratio_mean ) ) (mul (cmul (not has_pointer_deref) (mul num_paths) (mul (add (sub (mul (div num_ops dependence_height) ) exec_ratio) (sub (mul (cmul has_unsafe_jsr predict_product_mean ) (sub num_ops_max)) (sub (mul dependence_height_mean num_branches_max) num_paths))))))) GP Hyperblock Solutions General Purpose Intron that doesn’t affect solution

(add (sub (mul exec_ratio_mean ) ) (mul (cmul (not has_pointer_deref) (mul num_paths) (mul (add (sub (mul (div num_ops dependence_height) ) exec_ratio) (sub (mul (cmul has_unsafe_jsr predict_product_mean ) (sub num_ops_max)) (sub (mul dependence_height_mean num_branches_max) num_paths))))))) GP Hyperblock Solutions General Purpose Favor paths that don’t have pointer dereferences

GP Hyperblock Solutions General Purpose (add (sub (mul exec_ratio_mean ) ) (mul (cmul (not has_pointer_deref) (mul num_paths) (mul (add (sub (mul (div num_ops dependence_height) ) exec_ratio) (sub (mul (cmul has_unsafe_jsr predict_product_mean ) (sub num_ops_max)) (sub (mul dependence_height_mean num_branches_max) num_paths))))))) Favor highly parallel (fat) paths

GP Hyperblock Solutions General Purpose (add (sub (mul exec_ratio_mean ) ) (mul (cmul (not has_pointer_deref) (mul num_paths) (mul (add (sub (mul (div num_ops dependence_height) ) exec_ratio) (sub (mul (cmul has_unsafe_jsr predict_product_mean ) (sub num_ops_max)) (sub (mul dependence_height_mean num_branches_max) num_paths))))))) If a path calls a subroutine that may have side effects, penalize it

Case Study I: IMPACT’s Algorithm A B C E F G 4k24k 4k22k2k 25 28k D 10 Pathexechazopsdeppr A-B-D-F-G~ ~0 A-B-F-G A-C-F-G A-C-E-F-G A-C-E-G~ ~0

Case Study I: IMPACT’s Algorithm A B C E F G 4k24k 4k22k2k 25 28k D 10 Pathexechazopsdeppr A-B-D-F-G~ ~0 A-B-F-G A-C-F-G A-C-E-F-G A-C-E-G~ ~0