Model-Driven Generation of Performance Prototypes Steffen Becker FZI, Karlsruhe Tobias Dencker U Karlsruhe Jens Happe U Oldenburg DFG-Project PALLADIO

2 Generation of Performance Prototypes / SIPEW ‘08 / Steffen Becker / Motivation Software Development as Engineering 2 Image-Source: predictionprototypingtesting 20 ms23 ms25 ms System Model & Annotations Should Correspond, but does not because of Prediction Model’s Assumptions. How to close the gap?

3 Generation of Performance Prototypes / SIPEW ‘08 / Steffen Becker / Outline Prototype Generation Approaches Related Work Mapping PCM to Prototypes Generating Workload Workload Generator Calibration Realisation Business Information System Case Study Results Evaluation Summary Future Work Conclusions 3

4 Generation of Performance Prototypes / SIPEW ‘08 / Steffen Becker / Related Work 4 Capture Workload of Existing Application and Replay it in New Environment Delta: Base on Models, Early Design Time Performance Testing, Avritzer and Weyuker, 1996 Generate Artificial Workload based on LQN Model for CPU and Network Delta: Transformation of the Design Model, Data Dependent Workloads Synthetic Systems, Woodside and Schramm, 1996 MDABench: Generation of J2EE Protoypes from Models Delta: Manual Additions on Server Side needed MDABench, Zhou, Gorton, and Liu, 2006 Use Special Models Specifying Prototypes Delta: Additional Models for DB and Middleware Characterisation needed Test Bed Generator, Grundy, Cai et al., 2004

5 Generation of Performance Prototypes / SIPEW ‘08 / Steffen Becker / Palladio Component Model (PCM) Brief Overview 5

6 Generation of Performance Prototypes / SIPEW ‘08 / Steffen Becker / Mapping PCM to Prototypes PCM ConceptProtoCom Static InterfacesJava Interface BasicComponentsClasses with Simulated SEFF CompositeComponentsFacade Class AssemblyContextInstance of Component Class AssemblyConnectorDeployment Script Dynamic Internal ActionsResource Demand Generator Call ActionsRMI/SOAP Call Control FlowJava Control Flow Data Flow AnnotationsSimulated Dataflow Allocation AllocationContextDeployment Script Resources[Uses Physical Resources] Workload UsageModelWorkload Driver 6

7 Generation of Performance Prototypes / SIPEW ‘08 / Steffen Becker / Mapping PCM to Prototypes 7

8 Generation of Performance Prototypes / SIPEW ‘08 / Steffen Becker / Generating Load: Strategies  Load Generation should resemble target code as good as possible to capture hard to model effects  Different types of Load Generation Strategies needed CPU Intensive Load vs Memory Intensive Load Burst Disk Access vs Random Disk Reads Fragmented Network Transmission vs Bulks...  Can not assume Strategies have linear time demands  In this Presentation Calculating Fibonacci Numbers (CPU Intensive) vs Sorting Arrays (Memory Intensive) 8

9 Generation of Performance Prototypes / SIPEW ‘08 / Steffen Becker / Generating Load Calibration in Target Environment  Choose Algorithm whose time demand increases monotonic with a problem size n: alg(n)  Let exec alg (n) be the time demand for computing alg(n)  Given a time t dest find n such that t dest = exec alg (n) in case no contention is in the target environment  Problem: Needs calibration for a large set of t dest  Split t dest in power of 2 sum  Determine n i for with  Compute alg(n i ) for all x i = 1 9

10 Generation of Performance Prototypes / SIPEW ‘08 / Steffen Becker / Case Study: Input PCM Model 10 [Wu2003] 1,56s (80%) 6,47s (20%)

11 Generation of Performance Prototypes / SIPEW ‘08 / Steffen Becker / Case Study Setting 11 PCM Input Model generate Generated Prototype deploy & measure Dual Core PC Questions 1.Does it Generate, Compile and Deploy automatically? 2.Does Calibration work with different Algorithms? 3.Can we see hard-to-model Effects?

12 Generation of Performance Prototypes / SIPEW ‘08 / Steffen Becker / Case Study Q1 & Q2 Does Transformation and Calibration Work? 12 1,56s (80%) 6,47s (20%)

13 Generation of Performance Prototypes / SIPEW ‘08 / Steffen Becker / Case Study Q3 Unpredicted Effects? 13 1,56s (80%) 6,47s (20%)

14 Generation of Performance Prototypes / SIPEW ‘08 / Steffen Becker / Conclusions 14 MDSD Approach To Performance Prototyping Impact of Hard-to-Model Effects becomes Evaluatable Conclusions Additional Load Generators for Further Algorithms Further Types off Hardware (disk, network,...) Use Transformation Parameters (Mark Model) to Select Generator Strategies Empirically Test Performance Prediction Model’s Assumptions Future Work