1. PREDICTING UNROLL FACTORS USING SUPERVISED LEARNING
Mark Stephenson & Saman Amarasinghe
Massachusetts Institute of Technology
Computer Science and Artificial Intelligence Lab

2. INTRODUCTION & MOTIVATION
Compiler heuristics rely on detailed knowledge of the system
Compiler interactions not understood
Architectures are complex
Features
Pentium® (3M)
Pentium 4 (55M)
Superscalar 
Hyperthreading
Speculative execution
Improved FPU
This talk is really motivated by systems complexity.
To do a good job, the compiler has to have detailed knowledge about the system it is compiling to.
So things are getting harder, not easier.

3. HEURISTIC DESIGN
Current approach to heuristic development is somewhat ad hoc
Can compiler writers learn anything from baseball?
Is it feasible to deal with empirical data?
Can we use statistics and machine learning to build heuristics?
Just like baseball players, these systems are too complicated to model completely.

4. CASE STUDY Loop unrolling
Code expansion can degrade performance
Increased live ranges, register pressure
A myriad of interactions with other passes
Requires categorization into multiple classes
i.e., what’s the unroll factor?

5. ORC’S HEURISTIC (UNKNOWN TRIPCOUNT)
if (trip_count_tn == NULL) {
UINT32 ntimes = MAX(1, OPT_unroll_times-1);
INT32 body_len = BB_length(head);
while (ntimes > 1 && ntimes * body_len > CG_LOOP_unrolled_size_max)
ntimes--;
Set_unroll_factor(ntimes);
} else { … }

6. ORC’S HEURISTIC (KNOWN TRIPCOUNT)
} else {
BOOL const_trip = TN_is_constant(trip_count_tn);
INT32 const_trip_count = const_trip ? TN_value(trip_count_tn) : 0;
INT32 body_len = BB_length(head);
CG_LOOP_unroll_min_trip = MAX(CG_LOOP_unroll_min_trip, 1);
if (const_trip && CG_LOOP_unroll_fully &&
(body_len * const_trip_count <= CG_LOOP_unrolled_size_max ||
CG_LOOP_unrolled_size_max == 0 &&
CG_LOOP_unroll_times_max >= const_trip_count)) {
Set_unroll_fully();
Set_unroll_factor(const_trip_count);
UINT32 ntimes = OPT_unroll_times;
ntimes = MIN(ntimes, CG_LOOP_unroll_times_max);
if (!is_power_of_two(ntimes)) {
ntimes = 1 << log2(ntimes); }
while (ntimes > 1 && ntimes * body_len > CG_LOOP_unrolled_size_max)
ntimes /= 2;
if (const_trip) {
while (ntimes > 1 && const_trip_count < 2 * ntimes)
Set_unroll_factor(ntimes);
Our approach is going to be similar; we’re going to extract a bunch of these characteristics then let a ML algorithm learn the heuristics for us.

7. SUPERVISED LEARNING
Supervised learning algorithms try to find a function F(X) → Y
X : vector of characteristics that define a loop
Y : empirically found best unroll factor 1 2
Here I try to pictorially show how supervised learning works.
We start with a training set of examples, where examples contain a description of the loops and the empirically found best option for the loop.
We then let the learning algorithm find a function F(X) that tries to minimize the error on the training set.
Note that the mapping usually won’t be perfect because the learning algorithm will often try to sacrifice its performance on the training set to achieve a better generalization error.
Then, when the mapping is in place, we can predict the label for unseen examples.
I’ll talk more about the learning algorithms we use later. 3
Loops 4
Unroll Factors 5 6 7 8
F(X)

8. EXTRACTING THE DATA Extract features
Most features readily available in ORC
Kitchen sink approach
Finding the labels (best unroll factors)
Added instrumentation pass
Assembly instructions inserted to time loops
Calls to a library at all exit points
Compile and run at all unroll factors (1.. 8)
For each loop, choose the best one as the label
Features and labels are collected at the same time, and it is a fully automated process.

9. LEARNING ALGORITHMS Prototyped in Matlab
Two learning algorithms classified our data set well
Near neighbors
Support Vector Machine (SVM)
Both algorithms classify quickly
Train at the factory
No increase in compilation time
Matlab allowed us to use the plug and chug methodology. We downloaded – or the case of NN – wrote our own simple classifiers and quickly tested them out.

10. NEAR NEIGHBORS # FP operations # branches unroll don’t unroll
Here I show the operation of nearest neighbors for 2 classes. But, extending this to multiple classes is trivial – it works the same. This is a purely fabricated example.
Training the algorithm is as simple as populating a database.
The intuition behind this algorithm is that we want to treat similar loops similarly.
# branches
unroll
don’t unroll

11. SUPPORT VECTOR MACHINES
Map the original feature space into a higher-dimensional space (using a kernel)
Find a hyperplane that maximally separates the data

12. SUPPORT VECTOR MACHINES
# branches
# FP operations
# branches2
# FP operations 
unroll
don’t unroll

13. PREDICTION ACCURACY Leave-one-out cross validation
Filter out ambiguous training examples
Only keep obviously better examples (1.05x)
Throw away obviously noisy examples NN
SVM
ORC
Accuracy
62%
65%
16%
First order approximation of how well a classifier works as a predictor.

14. REALIZING SPEEDUPS (SWP DISABLED)
To stress the loop unroller we disabled SWP, otherwise all –O3 optimizations are performed.
Note that we couldn’t get fma3d to compile correctly with our instrumentation library. And we can’t compile eon because it is a c++ benchmark.
Similar to LOOCV, when we compile a benchmark we only use filtered training examples from the other benchmarks, excluding all examples from the benchmark being compiled.
5% overall, 9% for specfp benchmarks (perfect for spec is 12%).
The blue bars represent the best that we could hope to achieve. One note is that you can use our system as a glorified profiler.

15. FEATURE SELECTION
Feature selection is a way to identify the best features
Start with loads of features
Small feature sets are better
Learning algorithms run faster
Are less prone to overfitting the training data
Useless features can confuse learning algorithms

16. FEATURE SELECTION CONT. MUTUAL INFORMATION SCORE
Measures the reduction of uncertainty in one variable given knowledge of another variable
Does not tell us how features interact with each other
We use the kitchen sink approach to feature extraction – we extract all features that we think might be important. But learning algorithms often work better on smaller feature sets. Large feature sets can lead to overfitting the data, or unimportant features can flat-out confuse learning algorithms.
So we use feature selection to find the most informative features for us.
We can use MIS to tell us how informative a given feature is, or how well it can predict the unroll factor.

17. FEATURE SELECTION CONT. GREEDY FEATURE SELECTION
Choose single best feature
Choose another feature, that together with the best feature, improves classification accuracy most …

18. FEATURE SELECTION THE BEST FEATURES
Rank
Mutual Information Score
Greedy Feature Selection with SVM 1.
# FP operations
0.59 2.
# Operands
Loop nest level
0.49 3.
Instruction fan-in in DAG
0.34 4.
# Live ranges
# Branches
0.20 5.
# Memory operations
0.13

19. RELATED WORK
Monsifrot et al., “A Machine Learning Approach to Automatic Production of Compiler Heuristics.” 2002
Calder et al., “Evidence-Based Static Branch Prediction Using Machine Learning.” 1997
Cavazos et al., “Inducing Heuristic to Decide Whether to Schedule.” 2004
Moss et al., “Learning to Schedule Straight-Line Code.” 1997
Cooper et al., “Optimizing for Reduced Code Space using Genetic Algorithms.” 1999
Puppin et al., “Adapting Convergent Scheduling using Machine Learning.” 2003
Stephenson et al., “Meta Optimization: Improving Compiler Heuristics with Machine Learning.” 2003
Monsifrot: Supervised binary classification, ours is multiclass.

20. CONCLUSION
Supervised classification can effectively find good heuristics
Even for multi-class problems
SVM and near neighbors perform well
Potentially have big impact
Spent very little time tuning the learning parameters
Let a machine learning algorithm tell us which features are best

21. T H E N D

22. SOFTWARE PIPELINING ORC has been tuned with SWP in mind
Every major release of ORC has had a different unrolling heuristic for SWP
Currently 205 lines long
Can we learn a heuristic that outperforms ORC’s SWP unrolling heuristic?

23. REALIZING SPEEDUPS (SWP ENABLED)
We get a 1% improvement! But before you get down on our research, let me explain why this is the case. A big problem is noise. There are several cases where the oracle is outperformed by ORC or one of the learned classifiers. We assumed that loops were independent of each other, and this assumption is acceptable, but not totally correct. And the other reason is loop unrolling does not contribute much when SWP is enabled. The oracle, even though it is a bit noisy, only achieves a 4.5% speedup.
If you have long dependence chains, swp can still handle that. As long as you have enough iterations to fill the pipeline.
Control flow, procedure calls.
Dedicated hardware support.
Similar to LOOCV, when we compile a benchmark we only use filtered training examples from the other benchmarks, excluding all examples from the benchmark being compiled.

24. HURDLES Compiler writer must extract features
Acquiring labels takes time
Instrumentation library
~2 weeks to collect data
Predictions confined to training labels
Have to tweak learning algorithms
Noise
We believe that machine learning techniques have the potential to radically alter compiler construction methods. We learned a very small portion of a several million line compiler and got a 9% speedup on the specfps, so this approach shows promise. However, there are still many issues that need to be cleared up before this can become a truly viable technique.
First, the compiler writer must extract features, and this takes effort and may not always be easy. For us, most of the features that we used were readily available from the ORC infrastructure. We believe that as these techniques are shown to be viable, compiler passes will provide feature extraction tools, much like compiler infrastructures provide generic data flow analysis tools.
Also getting the labels takes time. The instrumentation library did take some effort, but again, in the future we expect timing utilities to be included in an infrastructure. Once the instrumentation is in place, this is a completely unsupervised process.
Predictions are confined to the labels with which the classifiers were trained. Future work will consider regression which does not have this limitation.
We’re using learning algorithms so that we don’t have to tweak the compiler heuristics, but the learning algorithms have parameters that must be tweaked. Once the training data set has been gathered, it’s pretty easy to write a meta-tuner that looks at a subset of the examples at tunes the learning parameters. Near neighbors has 1 parameter that has to be tuned, and the SVM has 2, so this is a much smaller search space than any compiler heuristic.
Noise is the single biggest detriment to our work. The finer the granularity at which you measure, the noisier the measurements will be. Modern architectures are helping our cause by including improved performance counters. Future research will explore ways to reduce noisy measurements.