1. KIT – University of the State of Baden-Wuerttemberg and National Research Center of the Helmholtz Association KIT – SOFTWARE DESIGN AND QUALITY GROUP www.kit.edu Generic Instrumentation and Monitoring Description for Software Performance Evaluation February 3, 2015 – ICPE 2015 – Austin Alexander Wert, Henning Schulz, Christoph Heger, Roozbeh Farahbod

2. KIT – Software Design and Quality Group 2/2202/03/2015 Measurement Environment Motivation Generic Instrumentation and Monitoring Description for Software Performance Evaluation System under Test Performance Expert Performance Problems? Performance Problems? Instrumentation & Monitoring Tool Performance Model? Performance Model? … … Performance Tests & Analysis Strategies Instrumentation and Monitoring Description Another IaM Tool Another SUT conceptually generic SUT specific IaM Tool specific SUT specific IaM Tool specific

3. KIT – Software Design and Quality Group 3/2202/03/2015 Idea Instrumentation & Monitoring Description Model (IDM)  Independent from specific: instrumentation and monitoring (IaM) tools system under tests programming languages (within the domain of modern-day, managed languages, e.g. Java,.NET, etc.) Generic Instrumentation and Monitoring Description for Software Performance Evaluation > IDM > IDM > IDM Instance > IDM Instance Instrumentation Engines System under Test Generic Concepts instance of interprets processes references instruments references contains implementations of

4. KIT – Software Design and Quality Group 4/2202/03/2015 Design of IDM Generic Instrumentation and Monitoring Description for Software Performance Evaluation Abstraction Orthogonality Composability Focus on Performance Response Time Memory Footprint CPU Time … CPU Utilization Network Utilization Memory Usage

5. KIT – Software Design and Quality Group 5/2202/03/2015 Instrumentation Scopes Generic Instrumentation and Monitoring Description for Software Performance Evaluation exclude include * *

6. KIT – Software Design and Quality Group 6/2202/03/2015 Conclusion Generic Instrumentation and Monitoring Description for Software Performance Evaluation Automatic Performance Problem Diagnostics Adaptable Instrumentation and Monitoring Instrumentation Description Model context independent light-weight composable So far used in: Future Work: integrate withMore probes and scopes extensibility concept

7. KIT – Software Design and Quality Group 7/2202/03/2015 Backup Generic Instrumentation and Monitoring Description for Software Performance Evaluation

8. KIT – Software Design and Quality Group 8/2202/03/2015 Related Work Aspect Oriented Instrumentation Languages AspectJ [Kiczales et al., 1997] DiSL [Marek et al., 2012] Josh [Chiba et al., 2004] SCoPE [Aotani et al., 2007]  language specific  no focus on performance Kieker [van Hoorn et al., 2012] uses AspectJ provides measurement probes for performance evaluation  measurement tool specific Generic Instrumentation and Monitoring Description for Software Performance Evaluation