1. Phase-Based Program Sampling Using Phoenix Chandra Krintz University of California, Santa Barbara Microsoft Faculty Summit July, 2005

2. Microsoft Faculty Summit, July 2005 2 Problem Statement Software is deployed Poorly optimized With bugs Minimally tested Goal of our work Efficient and accurate profiling of deployed software  Remote profiling  Include users in the process -- transparently To enable more effective software evolution  Couple dynamic compilation and software updating  Collect execution behavior to improve coverage testing Increasingly complex For a wide-range of devices

3. Microsoft Faculty Summit, July 2005 3 Problem Statement Software is deployed Poorly optimized With bugs Minimally tested Goal of our work Efficient and accurate profiling  Remote profiling  Include users in the process -- transparently To enable more effective software evolution  Couple dynamic compilation and software updating  Collect execution behavior to improve coverage testing Increasingly complex For a wide-range of devices

4. Microsoft Faculty Summit, July 2005 4 Our Phoenix-Based Research Efficient, flexible, and accurate profile collection Avoid computation/communication interruption Gather important profile types (paths, data, methods…) Our extensions to Phoenix: 1. Turn profiling on and off dynamically Program execution sampling 2. Trigger sampling to enable Highly accurate profiles (similar to exhaustive profiles) With very low overhead (sample very seldomly)

5. Microsoft Faculty Summit, July 2005 5 Toggling Profile Collection When to sample the executing program Periodically Randomly According to program behavior

6. Microsoft Faculty Summit, July 2005 6 Toggling Profile Collection When to sample the executing program According to program behavior Method entry Method call

7. Microsoft Faculty Summit, July 2005 7 Sample According to Program Behavior Duplicate code One copy is instrumented, one is not Alternate execution between them [Arnold’01] Method entry Method entry Method call

8. Microsoft Faculty Summit, July 2005 8 Toggling Profile Collection Duplicate code One copy is instrumented, one is not Alternate execution between them Method entry Method entry Threshold-based counters Method call

9. Microsoft Faculty Summit, July 2005 9 Toggling Profile Collection Duplicate code One copy is instrumented, one is not Alternate execution between them Method entry Method entry Method call Control transfers to uninstrumented code Control transfers to instrumented code

10. Microsoft Faculty Summit, July 2005 10 Toggling Profile Collection Duplicate code One copy is instrumented, one is not Alternate execution between them Method entry Method entry Method call Control transfers to uninstrumented code Control transfers to instrumented code Can also stay in instrumented code for longer periods of time (based on thresholds), i.e., bursts [Hirzel’01,Chilimbi’04]

11. Microsoft Faculty Summit, July 2005 11 Toggling Profile Collection Phoenix implementation Duplicate code within the same method [Arnold’01] Insert instrumentation in one copy Insert branches that jump between the two  Based on counters Users can Specify the type of profile to collect Threshold values  Global or local  Control sampling frequency and burst length

12. Microsoft Faculty Summit, July 2005 12 Toggle Triggers Currently, based on counters However, this may cause us to collect redundant information or miss important activities Our second focus is an intelligent sampling trigger Based on program phases [Sherwood’03]  Repeating patterns in program behavior

13. Microsoft Faculty Summit, July 2005 13 Toggle Triggers Currently, based on counters However, this may cause us to collect redundant information or miss important activities Our second focus is an intelligent sampling trigger Based on program phases [Sherwood’03]  Repeating patterns in program behavior

14. Microsoft Faculty Summit, July 2005 14 Toggle Triggers Currently, based on counters However, this may cause us to collect redundant information or miss important activities Extant approaches to sampling: Periodic sampling

15. Microsoft Faculty Summit, July 2005 15 Toggle Triggers Currently, based on counters However, this may cause us to collect redundant information or miss important activities Extant approaches to sampling: Random sampling

16. Microsoft Faculty Summit, July 2005 16 Toggle Triggers Currently, based on counters However, this may cause us to collect redundant information or miss important activities Extant approaches to sampling: Bursty sampling

17. Microsoft Faculty Summit, July 2005 17 Toggle Triggers Currently, based on counters However, this may cause us to collect redundant information or miss important activities Our second focus is an intelligent sampling trigger Based on program phases [Nagpurkar’05] Phase-base sampling 5% Error = 55-80% impr.

18. Microsoft Faculty Summit, July 2005 18 Efficacy of Phase Profiling % sampled vs average error across benchmarks Error = difference between sampled profile and exhaustive profile At 5% error: equates to a 55-80% reduction in overhead 05101520 25 % Program Sampled 0 10 20 30 40 % Error in Block Counts avg random max random periodic phaseaware

19. Microsoft Faculty Summit, July 2005 19 Conclusions Phoenix provides a framework for program instrumentation, analysis, and optimization Our work Extends Phoenix to enable program sampling  That is parameterizable (frequency, burst length)  Parameters can be changed dynamically Extends sampling to be aware of program behavior  Phase-aware  Enables collection of accurate profiles with very little overhead  55-80% overhead reduction over periodic/random sampling Provides a visualization tool (consumes phase profiles)  Enables users to visualize/reason about program phase behavior

20. Microsoft Faculty Summit, July 2005 20 Acknowledgements Students involved DirectlyIndirectly Lingli Zhang (prof-guided opt) Sunil Soman (garbage collection) Ricky Tiang (profiling)Hussam Mousa (profile toggling) Priya Nagpurkar (phase profiling)Selim Gurun (embedded systems) Thanks! More Info: Microsoft UCSB RACELab John Lefor www.cs.ucsb.edu/~racelab Vinod Grover Phoenix Development Team