1. Computer & Information Sciences - University of Delaware Colloquium / 55 Exhaustive Phase Order Search Space Exploration and Evaluation by Prasad Kulkarni (Florida State University)

2. Computer & Information Sciences - University of Delaware Colloquium / 552 Compiler Optimizations To improve efficiency of compiler generated code Optimization phases require enabling conditions –need specific patterns in the code –many also need available registers Phases interact with each other Applying optimizations in different orders generates different code

3. Computer & Information Sciences - University of Delaware Colloquium / 553 Phase Ordering Problem To find an ordering of optimization phases that produces optimal code with respect to possible phase orderings Evaluating each sequence involves compiling, assembling, linking, execution and verifying results Best optimization phase ordering depends on –source application –target platform –implementation of optimization phases Long standing problem in compiler optimization!!

4. Computer & Information Sciences - University of Delaware Colloquium / 554 Phase Ordering Space Current compilers incorporate numerous different optimization phases –15 distinct phases in our compiler backend 15! = 1,307,674,368,000 Phases can enable each other –any phase can be active multiple times 15 15 = 437,893,890,380,859,375 –cannot restrict sequence length to 15 15 44 = 5.598 * 10 51

5. Computer & Information Sciences - University of Delaware Colloquium / 555 Addressing Phase Ordering Exhaustive Search –universally considered intractable We are now able to exhaustively evaluate the optimization phase order space.

6. Computer & Information Sciences - University of Delaware Colloquium / 556 Re-stating of Phase Ordering Earlier approach –explicitly enumerate all possible optimization phase orderings Our approach –explicitly enumerate all function instances that can be produced by any combination of phases

7. Computer & Information Sciences - University of Delaware Colloquium / 557 Outline Experimental framework Exhaustive phase order space evaluation Faster conventional compilation Conclusions Summary of my other work Future research directions

8. Computer & Information Sciences - University of Delaware Colloquium / 558 Outline Experimental framework Exhaustive phase order space evaluation Faster conventional compilation Conclusions Summary of my other work Future research directions

9. Computer & Information Sciences - University of Delaware Colloquium / 559 Experimental Framework We used the VPO compilation system –established compiler framework, started development in 1988 –comparable performance to gcc –O2 VPO performs all transformations on a single representation (RTLs), so it is possible to perform most phases in an arbitrary order Experiments use all the 15 re-orderable optimization phases in VPO Target architecture was the StrongARM SA-100 processor

10. Computer & Information Sciences - University of Delaware Colloquium / 5510 VPO Optimization Phases IDOptimization PhaseIDOptimization Phase bbranch chaininglloop transformations ccommon subexpr. elim.ncode abstraction dremv. unreachable codeoeval. order determin. gloop unrollingqstrength reduction hdead assignment elim.rreverse branches iblock reorderingsinstruction selection jminimize loop jumpsuremv. useless jumps kregister allocation

11. Computer & Information Sciences - University of Delaware Colloquium / 5511 Disclaimers Did not include optimization phases normally associated with compiler front ends –no memory hierarchy optimizations –no inlining or other interprocedural optimizations Did not vary how phases are applied Did not include optimizations that require profile data

12. Computer & Information Sciences - University of Delaware Colloquium / 5512 Benchmarks 12 MiBench benchmarks; 244 functions CategoryProgramDescription auto bitcounttest processor bit manipulation abilities qsortsort strings using quicksort sorting algorithm network dijkstraDijkstra’s shortest path algorithm patriciaconstruct patricia trie for IP traffic telecomm fftfast fourier transform adpcmcompress 16-bit linear PCM samples to 4-bit consumer jpegimage compression and decompression tiff2bwconvert color.tiff image to b&w image security shasecure hash algorithm blowfishsymmetric block cipher with variable length key office stringsearchsearches for given words in phrases ispellfast spelling checker

13. Computer & Information Sciences - University of Delaware Colloquium / 5513 Outline Experimental framework Exhaustive phase order space evaluation Faster conventional compilation Conclusions Summary of my other work Future research directions

14. Computer & Information Sciences - University of Delaware Colloquium / 5514 Terminology Active phase – An optimization phase that modifies the function representation Dormant phase – A phase that is unable to find any opportunity to change the function Function instance – any semantically, syntactically, and functionally correct representation of the source function (that can be produced by our compiler)

15. Computer & Information Sciences - University of Delaware Colloquium / 5515 Naïve Optimization Phase Order Space All combinations of optimization phase sequences are attempted a b c d a bc dadadad bcbcbc L2 L1 L0

16. Computer & Information Sciences - University of Delaware Colloquium / 5516 Eliminating Consecutively Applied Phases A phase just applied in our compiler cannot be immediately active again a b c d bc dadada cbbc L2 L1 L0 a bc d

17. Computer & Information Sciences - University of Delaware Colloquium / 5517 Eliminating Dormant Phases Get feedback from the compiler indicating if any transformations were successfully applied in a phase. L2 L1 L0 a b c d bc dadad cb a bc

18. Computer & Information Sciences - University of Delaware Colloquium / 5518 Identical Function Instances Some optimization phases are independent –example: branch chaining & register allocation Different phase sequences can produce the same code r[2] = 1; r[2] = 1; r[3] = r[4] + r[2]; r[3] = r[4] + r[2];  instruction selection r[3] = r[4] + 1; r[3] = r[4] + 1; r[2] = 1; r[2] = 1; r[3] = r[4] + r[2]; r[3] = r[4] + r[2];  constant propagation r[2] = 1; r[2] = 1; r[3] = r[4] + 1; r[3] = r[4] + 1;  dead assignment elimination r[3] = r[4] + 1; r[3] = r[4] + 1;

19. Computer & Information Sciences - University of Delaware Colloquium / 5519 Equivalent Function Instances sum = 0; for (i = 0; i < 1000; i++ ) sum += a [ i ]; Source Code r[10]=0; r[12]=HI[a]; r[12]=r[12]+LO[a]; r[1]=r[12]; r[9]=4000+r[12]; L3 r[8]=M[r[1]]; r[10]=r[10]+r[8]; r[1]=r[1]+4; IC=r[1]?r[9]; PC=IC<0,L3; Register Allocation before Code Motion r[11]=0; r[10]=HI[a]; r[10]=r[10]+LO[a]; r[1]=r[10]; r[9]=4000+r[10]; L5 r[8]=M[r[1]]; r[11]=r[11]+r[8]; r[1]=r[1]+4; IC=r[1]?r[9]; PC=IC<0,L5; Code Motion before Register Allocation r[32]=0; r[33]=HI[a]; r[33]=r[33]+LO[a]; r[34]=r[33]; r[35]=4000+r[33]; L01 r[36]=M[r[34]]; r[32]=r[32]+r[36]; r[34]=r[34]+4; IC=r[34]?r[35]; PC=IC<0,L01; After Mapping Registers

20. Computer & Information Sciences - University of Delaware Colloquium / 5520 Efficient Detection of Unique Function Instances After pruning dormant phases there may be tens or hundreds of thousands of unique instances Use a CRC (cyclic redundancy check) checksum on the bytes of the RTLs representing the instructions Used a hash table to check if an identical or equivalent function instance already exists in the DAG

21. Computer & Information Sciences - University of Delaware Colloquium / 5521 Eliminating Identical/Equivalent Function Instances Resulting search space is a DAG of function instances L2 L1 L0 a b c c d a d a d

22. Computer & Information Sciences - University of Delaware Colloquium / 5522 Static Enumeration Results FunctionInst.Fn_instLenCF Batch Vs. optimal start_input_bmp (j)1,372120,77725701.41 correct(i)1,2951,348,154256634.18 main (t)1,2762,882,0212938916.25 parse_switches (j) 1,228180,76220530.41 start_input_gif (j) 100939,35221182.46 start_input_tga (j) 97263,45821301.66 askmode (i)942232,453241087.87 skiptoword (i)901439,994221031.45 start_input_ppm (j)7958,52116452.70 Average (234)196.289,946.714.736.26.46

23. Computer & Information Sciences - University of Delaware Colloquium / 5523 Exhaustively enumerated the optimization phase order space to find an optimal phase ordering with respect to code-size [Published in CGO ’06]

24. Computer & Information Sciences - University of Delaware Colloquium / 5524 Determining Program Performance Almost 175,000 distinct function instances, on average –largest enumerated function has 2,882,021 instances Too time consuming to execute each distinct function instance –assemble  link  execute more expensive than compilation Many embedded development environments use simulation –simulation orders of magnitude more expensive than execution Use data obtained from a few executions to estimate the performance of all remaining function instances

25. Computer & Information Sciences - University of Delaware Colloquium / 5525 Determining Program Performance (cont...) Function instances having identical control-flow graphs execute each block the same number of times Execute application once for each control- flow structure Statically estimate the number of cycles required to execute each basic block dynamic frequency measure =  static cycles * block frequency)

26. Computer & Information Sciences - University of Delaware Colloquium / 5526 Predicting Relative Performance – I 4 cycles 20 cycles 27 cycles 2 cycles 5 cycles 10 cycles 20 15 20 5 5 2 4 cycles 22 cycles 25 cycles 2 cycles 10 cycles 20 15 20 5 5 2 Total cycles = 744 Total cycles = 789

27. Computer & Information Sciences - University of Delaware Colloquium / 5527 Dynamic Frequency Results FunctionInst.Fn_instLenCFLeaf % from optimal BatchWorst main (t)1,2762,882,0212938915,1640.084.3 parse_switches (j) 1,228180,76220532,0276.764.8 askmode (i)942232,453241084758.456.2 skiptoword (i)901439,994221032,8346.149.6 start_input_ppm (j)7958,5211645801.728.4 pfx_list_chk (i)6401,269,638441364,6604.378.6 main (f)6242,789,903331224,2147.546.1 sha_transform (h)541548,81232985,2629.6133.4 main (p)48314,51015101787.713.1 Average (79)234.3174,574.816.147.4813.44.865.4

28. Computer & Information Sciences - University of Delaware Colloquium / 5528 Correlation – Dynamic Frequency Counts Vs. Simulator Cycles Static performance estimation is inaccurate –ignored cache/branch misprediction penalties Most embedded systems have simpler architectures –estimation may be sufficiently accurate –simulator cycles are close to executed cycles We show strong correlation between our measure of performance and simulator cycles

29. Computer & Information Sciences - University of Delaware Colloquium / 5529 Complete Function Correlation Example: init_search in stringsearch

30. Computer & Information Sciences - University of Delaware Colloquium / 5530 Leaf Function Correlation Leaf function instances are generated when no additional phases can be successfully applied Leaf instances provide a good sampling –represents the only code that can be generated by an aggressive compiler, like VPO –at least one leaf instance represents an optimal phase ordering for over 86% of functions –significant percent of leaf instances among optimal

31. Computer & Information Sciences - University of Delaware Colloquium / 5531 Leaf Function Correlation Statistics Pearson’s correlation coefficient – Accuracy of our estimate of optimal perf. –  xy – (  x  y)/n sqrt( (  x 2 – (  x) 2 /n) * (  y 2 - (  y) 2 /n) ) Pcorr = Lcorr = cycle count for best leaf cy. cnt for leaf with best dynamic freq count

32. Computer & Information Sciences - University of Delaware Colloquium / 5532 Leaf Function Correlation Statistics (cont…) FunctionPcorr Lcorr 0%Lcorr 1% RatioLeavesRatioLeaves AR_btbl...(b)1.00 1 1 BW_btbl...(b)1.00 2 2 bit_count.(b)1.00 2 2 bit_shifter(b)1.00 2 2 bitcount(b)0.890.921 1 main(b)1.00 6 23 ntbl_bitcnt(b)1.000.952 2 ntbl_bit…(b)0.991.002 2 dequeue(d)0.991.006 6 dijkstra(d)1.000.9741.00269....…. average0.960.984.380.99621

33. Computer & Information Sciences - University of Delaware Colloquium / 5533 Exhaustively evaluated the optimization phase order space to find a near-optimal phase ordering with respect to simulator cycles [Published in LCTES ’06]

34. Computer & Information Sciences - University of Delaware Colloquium / 5534 Outline Experimental framework Exhaustive phase order space evaluation Faster conventional compilation Conclusions Summary of my other work Future research directions

35. Computer & Information Sciences - University of Delaware Colloquium / 5535 Phase Enabling Interaction b enables a along the path a-b-a a b c b accb ad

36. Computer & Information Sciences - University of Delaware Colloquium / 5536 Phase Enabling Probabilities Ph St b c d g h i j k l n o q r s u b0.72 0.02 0.010.04 0.01 0.02 0.66 c1.000.01 0.680.010.020.070.050.150.34 d 1.00 g0.220.280.170.050.020.140.340.090.15 h0.08 0.16 0.140.020.01 0.20 i0.720.04 0.01 0.09 j0.030.06 0.44 k 0.98 0.280.01 0.020.01 0.96 l0.60 0.73 0.02 0.01 0.030.53 n0.41 0.36 0.01 0.29 o0.88 0.40 0.03 q 0.99 0.02 0.99 r0.570.06 s1.00 0.33 0.41 0.830.070.050.150.07 u 0.01 0.02

37. Computer & Information Sciences - University of Delaware Colloquium / 5537 Phase Disabling Interaction b disables a along the path b-c-d a b c b accb ad

38. Computer & Information Sciences - University of Delaware Colloquium / 5538 Disabling Probabilities Ph b c d g h i j k l n o q r s u b1.00 0.28 0.090.18 0.20 0.11 0.01 c 1.00 0.02 0.080.020.300.321.00 0.08 d 1.00 0.03 0.01 g0.13 1.00 0.060.01 0.12 0.22 h0.01 1.00 0.040.101.00 0.01 i0.02 0.22 1.000.20 0.01 0.44 0.91 j 0.01 0.081.00 0.01 0.16 k 0.010.05 1.000.050.141.00 l0.021.000.110.04 0.071.000.321.00 n0.070.01 0.020.01 1.00 0.01 o 0.080.011.00 q r0.06 0.200.36 1.00 0.05 s 0.07 0.03 0.310.220.140.260.021.00 u0.41 0.02 0.340.15 1.00

39. Computer & Information Sciences - University of Delaware Colloquium / 5539 Faster Conventional Compiler Modified VPO to use enabling and disabling phase probabilities to decrease compilation time # p[i] - current probability of phase i being active # e[i][j] - probability of phase j enabling phase i # d[i][j] - probability of phase j disabling phase i For each phase i do p[i] = e[i][st]; While (any p[i] > 0) do Select j as the current phase with highest probability of being active Apply phase j If phase j was active then For each phase i, where i != j do p[i] += ((1-p[i]) * e[i][j]) - (p[i] * d[i][j]) p[j] = 0

40. Computer & Information Sciences - University of Delaware Colloquium / 5540 Probabilistic Compilation Results FunctionOld CompilationProb. CompilationProb. / Old AttemptedActiveAttemptedActiveTimeSizeSpeed start_inp...(j)2331655140.4691.014 N/A parse_swi...(j)2331453120.3711.016 0.972 start_inp...(j)2701555140.3531.010 N/A start_inp...(j)2331449130.4201.003 N/A start_inp...(j)2311153120.4361.004 1.000 fft_float(f)4632899250.4511.012 0.974 main(f)2842073180.5501.007 1.000 sha_trans...(h)2841767160.6050.965 0.953 read_scan...(j)2331343100.3421.018 N/A LZWReadByte(j)2681245110.3251.014 N/A main(j)2701257140.3751.007 1.000 dijkstra(d)23194390.4091.010 1.000.... average230.38.947.79.60.2971.015 1.005

41. Computer & Information Sciences - University of Delaware Colloquium / 5541 Outline Experimental framework Exhaustive phase order space evaluation Faster conventional compilation Conclusions Summary of my other work Future research directions

42. Computer & Information Sciences - University of Delaware Colloquium / 5542 Conclusions Phase ordering problem –long standing problem in compiler optimization –exhaustive evaluation always considered infeasible Exhaustively evaluated the phase order space –re-interpretation of the problem –novel application of search algorithms –fast pruning techniques –accurate prediction of relative performance Analyzed properties of the phase order space to speedup conventional compilation published in CGO’06, LCTES’06, submitted to TOPLAS

43. Computer & Information Sciences - University of Delaware Colloquium / 5543 Challenges Exhaustive phase order search is a severe stress test for the compiler –isolate analysis required and invalidated by each phase –produce correct code for all phase orderings –eliminate all memory leaks Search algorithm needs to be highly efficient –used CRCs and hashes for function comparisons –stored intermediate function instances to reduce disk access –maintained logs to restart search after crash

44. Computer & Information Sciences - University of Delaware Colloquium / 5544 Outline Experimental framework Exhaustive phase order space evaluation Faster conventional compilation Conclusions Summary of my other work Future research directions

45. Computer & Information Sciences - University of Delaware Colloquium / 5545 VISTA Provides an interactive code improvement paradigm –view low-level program representation –apply existing phases and manual changes in any order –browse and undo previous changes –automatically obtain performance information –automatically search for effective phase sequences Useful as a research as well as teaching tool –employed in three universities published in LCTES ’03, TECS ‘06

46. Computer & Information Sciences - University of Delaware Colloquium / 5546 VISTA – Main Window

47. Computer & Information Sciences - University of Delaware Colloquium / 5547 Faster Genetic Algorithm Searches Improving performance of genetic algorithms –avoid redundant executions of the application over 87% of executions were avoided reduce search time by 62% –modify search to obtain comparable results in fewer generations reduced GA generations by 59% reduce search time by 35% published in PLDI ’04, TACO ’05

48. Computer & Information Sciences - University of Delaware Colloquium / 5548 Heuristic Search Algorithms Analyzing the phase order space to improve heuristic algorithms –detailed performance and cost comparison of different heuristic algorithms –demonstrated the importance and difficulty of selecting the correct sequence length –illustrated the importance of leaf function instances –proposed modifications to existing algorithms, and new search algorithms Will be published in CGO ‘07

49. Computer & Information Sciences - University of Delaware Colloquium / 5549 Dynamic Compilation Explored asynchronous dynamic compilation in a virtual machine –demonstrated shortcomings of current popular compilation strategy –describe importance of minimum compiler utilization –discussed new compilation strategies –explored the changes needed to current compilation strategies to exploit free cycles Submitted to VEE ‘07

50. Computer & Information Sciences - University of Delaware Colloquium / 5550 Outline Experimental framework Exhaustive phase order space evaluation Faster conventional compilation Conclusions Summary of my other work Future research directions

51. Computer & Information Sciences - University of Delaware Colloquium / 5551 Support for parallelism –traditional languages –express parallelism –dynamic scheduling Virtual machines –dynamic code generation and optimization Push compilation decisions further down Compiler Technology Challenges c omp i le r multi-core –heterogeneous cores No great solution –performance monitoring –software-controlled reconfiguration Can no longer do it alone High Level Language Machine Architecture

52. Computer & Information Sciences - University of Delaware Colloquium / 5552 Iterative Compilation & Machine Learning Improved scope for iterative compilation & machine learning –proliferation of new architectures automate tuning compiler heuristics –tuning important libraries –using performance monitors dynamic JIT compilers How to use machine learning to optimize and schedule more efficiently ?

53. Computer & Information Sciences - University of Delaware Colloquium / 5553 Dynamic Compilation Virtual machines likely to grow in importance –productivity, portability, interoperability, isolation... Challenges –when, what, how to parallelize –using hardware performance monitors –using static analyses to aid dynamic compilation –debugging tools for correctness and performance debugging

54. Computer & Information Sciences - University of Delaware Colloquium / 5554 Heterogeneous Multi-core Architectures Can provide the best performance, cost, power balance Challenges –schedule tasks, allocate resources –dynamic core-specific optimization –automatic data layout to prevent conflicts

55. Computer & Information Sciences - University of Delaware Colloquium / 5555 Questions ?

56. Computer & Information Sciences - University of Delaware Colloquium / 5556 Results – Code Size Summary Exhaustively enumerated 234 out of 244 functions Sequence length –Maximum: 44; Average: 14.71 Distinct function instances –Maximum: 2,882,021; Average: 89,947 Distinct control-flows –Maximum: 920; Average: 36.2 Code size improvement over default sequence –Maximum: 63%; Average: 6.46%

57. Computer & Information Sciences - University of Delaware Colloquium / 5557 Results – Performance Summary Exhaustively evaluated 79 out of 88 functions Sequence length –Maximum: 44; Average: 16.1 Distinct function instances –Maximum: 2,882,021; Average: 174,574.8 Distinct control-flows –Maximum: 920; Average: 47.4 Performance improvement over default sequence –Maximum: 15%; Average: 4.8%

58. Computer & Information Sciences - University of Delaware Colloquium / 5558 Leaf Vs. Non-Leaf Performance

59. Computer & Information Sciences - University of Delaware Colloquium / 5559 Phase Order Space Evaluation – Summary generatenextoptimizationsequence lastphaseactive?identicalfunctioninstance?equivalentfunctioninstance? calculatefunctionperformancesimulateapplicationseencontrol-flowstructure? Y YY Y N NN N add node to DAG

60. Computer & Information Sciences - University of Delaware Colloquium / 5560 Phase Order Space Evaluation – Summary lastphaseactive?identicalfunctioninstance?equivalentfunctioninstance? calculatefunctionperformancesimulateapplicationseencontrol-flowstructure? Y YY Y N NN N add node to DAG generatenextoptimizationsequence

61. Computer & Information Sciences - University of Delaware Colloquium / 5561 Phase Order Space Evaluation – Summary lastphaseactive?identicalfunctioninstance?equivalentfunctioninstance? calculatefunctionperformancesimulateapplicationseencontrol-flowstructure? Y YY Y N NN N add node to DAG generatenextoptimizationsequence

62. Computer & Information Sciences - University of Delaware Colloquium / 5562 Phase Order Space Evaluation – Summary lastphaseactive?identicalfunctioninstance?equivalentfunctioninstance? calculatefunctionperformancesimulateapplicationseencontrol-flowstructure? Y YY Y N NN N add node to DAG generatenextoptimizationsequence

63. Computer & Information Sciences - University of Delaware Colloquium / 5563 Phase Order Space Evaluation – Summary lastphaseactive?identicalfunctioninstance?equivalentfunctioninstance? calculatefunctionperformancesimulateapplicationseencontrol-flowstructure? Y YY Y N NN N add node to DAG generatenextoptimizationsequence

64. Computer & Information Sciences - University of Delaware Colloquium / 5564 Phase Order Space Evaluation – Summary lastphaseactive?identicalfunctioninstance?equivalentfunctioninstance? calculatefunctionperformancesimulateapplicationseencontrol-flowstructure? Y YY Y N NN N add node to DAG generatenextoptimizationsequence

65. Computer & Information Sciences - University of Delaware Colloquium / 5565 Phase Order Space Evaluation – Summary lastphaseactive?identicalfunctioninstance?equivalentfunctioninstance? calculatefunctionperformancesimulateapplicationseencontrol-flowstructure? Y YY Y N NN N add node to DAG generatenextoptimizationsequence

66. Computer & Information Sciences - University of Delaware Colloquium / 5566 Phase Order Space Evaluation – Summary lastphaseactive?identicalfunctioninstance?equivalentfunctioninstance? calculatefunctionperformancesimulateapplicationseencontrol-flowstructure? Y YY Y N NN N add node to DAG generatenextoptimizationsequence

67. Computer & Information Sciences - University of Delaware Colloquium / 5567 Weighted Function Instance DAG Each node is weighted by the number of paths to a leaf node 5 1 1 1 22 11 1 [abc] [bc] [c][ab] [d] [a] a b c b accb ad

68. Computer & Information Sciences - University of Delaware Colloquium / 5568 Predicting Relative Performance – II 4 cycles 10 cycles 5 4 cycles ? 15 cycles 26 cycles 15 cycles 90 cycles 2 cycles 44 cycles 10 cycles ? ? ? ? ? ? ? Total cycles = 170 Total cycles = ?? 10 cycles 5 5 5

69. Computer & Information Sciences - University of Delaware Colloquium / 5569 Case when No Leaf is Optimal