1. 1 © NOKIA Nokia Research Center / 18.05.04 Performance Data Collection: Hybrid Approach Edu Metz, Raimondas Lencevicius Software Performance Architecture Group Nokia Research Center 5 Wayside Road, Burlington, MA 01803, USA E-mail: Edu.Metz@nokia.com, Raimondas.Lencevicius@nokia.com

2. 2 © NOKIA Nokia Research Center / 18.05.04 Motivation Performance profiling of complex embedded software is important Profiling = monitoring software execution + obtained data analysis 2 ways to collect data Event tracing Sampling Both have drawbacks and advantages What to use?

3. 3 © NOKIA Nokia Research Center / 18.05.04 Position Data collection should be performed using hybrid approach that combines completeness of event tracing with low cost of sampling

4. 4 © NOKIA Nokia Research Center / 18.05.04 Overview Event tracing Sampling Hybrid data collection with example Why it will fail Why it will succeed

5. 5 © NOKIA Nokia Research Center / 18.05.04 Event Tracing Record events occurring during system execution Component entry and exit Function calls Execution states Message communication Resource usage Complete record of information If event was recorded, it did occur; if it was not recorded, it did not occur Allows complete analysis of event relationships in time Event tracing comes at a cost Trace instrumentation takes time Changes behavior of software system Could violate real-time constraints and timing requirements in real-time systems Disproportional overhead for different software components Large data volume

6. 6 © NOKIA Nokia Research Center / 18.05.04 Sampling Intermittent recording of software system’s state Program counter Call stack Scheduled and blocked tasks Can be done strictly periodically or with certain randomness Overhead may be orders of magnitude less than event tracing Has its own downsides Granularity of information determined by sampling frequency Needs “reasonable” data collection interval to provide accurate data Loss of information completeness May complicate application-level information extraction

7. 7 © NOKIA Nokia Research Center / 18.05.04 Hybrid Approach Often a way to minimize drawbacks of two techniques is to use them both! Trace a subset of events Sample the rest Provide low overhead with adequate information completeness But… how not to inherit drawbacks of two techniques? Worst case: high overhead and not enough information for causality determination Intelligent selection of what should be traced and what should be sampled is needed

8. 8 © NOKIA Nokia Research Center / 18.05.04 Hybrid Approach Example: Profiling with Tasks System contains a number of communicating tasks Performance engineer needs to understand what parts of software consume most of the processor time and how this influences task messaging and scheduling Using only event tracing would lead to large overhead Using sampling would not provide complete information about task messaging Hybrid approach solves the problem Low overhead due to sampling of inside task activities Complete information of task messaging through event tracing - low overhead due to infrequent tracing

9. 9 © NOKIA Nokia Research Center / 18.05.04 Hybrid Approach Heuristics What should be traced? Infrequent events Task switches Peripheral activations and deactivations Events that have causality relationships Message sends and receives Interrupts (maybe) Events for which relative time to other event is necessary Video frame display Events for which the absolute time is necessary Events connected with real world: alarm clock What should be sampled? Frequent events Function calls (maybe) Bytecode/CPU op execution Events for which causality is not important Heap size (maybe)

10. 10 © NOKIA Nokia Research Center / 18.05.04 Related Work We do not know any hybrid profilers A lot of sampling profilers Intel Vtune – either sample or trace Hollingsworth et al. – collect all and then sample the collection buffers periodically

11. 11 © NOKIA Nokia Research Center / 18.05.04 Why it will Fail Intelligent selection of what should be traced and what should be sampled is difficult Bad tracing/sampling selection may lead to large overhead and missing information Trace instrumentation is still required with all its drawbacks Yields two sets of profiling data that need to be reconciled

12. 12 © NOKIA Nokia Research Center / 18.05.04 Why it will Succeed Solves the issue of obtaining complete causality information with low overhead Allows collecting infrequent events even in short program runs Limits profiling data volume

13. 13 © NOKIA Nokia Research Center / 18.05.04 Questions ?