1. Performance evaluation of component-based software systems Seminar of Component Engineering course Rofideh hadighi rhadighi@ustmb.ac.ir 7 Jan 2010

2. Outline Introduction Why performance evaluation? Software component performance Performance evaluation methods Evaluation Related work Future directions Conclusions 2 Performance evaluation of component-based software systems

3. Software components: ◦ Units of composition with defined:  Provided and required interfaces Component-based software engineering: ◦ Successorofobject-oriented software development The challenge for component performance models: ◦ Performance of software component in running system:  Context  Usage profile  Usually unknown to the component developer 3 Performance evaluation of component-based software systems Introduction

4. Why performance evaluation? Analyzing models: o Avoid performance problems in implementation o Avoid substantial costs for redesigning. Analyze the observable performance : o Implemented and running components.  To understand performance properties.  To determine their maximum capacity.  Identify performance-critical components.  To remove performance bottlenecks. 4 Performance evaluation of component-based software systems

5. Software component performance 1. Factors influencingcomponent performance. 2. Component life-cycle. 3. Requirements for a component performance modeling language. Performance evaluation of component-based software systems 5

6. Factors influencing component performance: Performance evaluation of component-based software systems 6 Fig1. Factors influencing component performance [1]

7. 1.Factors influencing component performance: a)Component implementation:  Different implementation:  Two components provide same service functionally.  Exhibit different execution times.  Running on the same resources.  Given the same inputs. b)Required services:  Component service A invokes required services B:  The execution time of B adds up to the execution time of A.  overall execution time of a component service:  execution time of required services. Performance evaluation of component-based software systems 7

8. 1.Factors influencing component performance: (cont`d) c)Deployment platform: ◦ Deploy software component to different platforms. ◦ A deployment platform may include:  Several software layers:  Component container, virtual machine, operating system.  Several hardware:  Processor, storage device, network. Performance evaluation of component-based software systems8

9. d)Usage profile: ◦ Clients invoke component services:  Different input parameters: ◦ The execution time of a service can change depending:  Values of the input parameters.  Receive parameters as the result of calls to required services. e)Resource contention: ◦ Software component  Doesn't execute as a single process in isolation on a given platform. ◦ Induced waiting times for accessing limited resources:  Aadd up to the execution time. Performance evaluation of component-based software systems9 1.Factors influencing component performance: (cont`d)

10. 2. Component life-cycle. a) A specified component: ◦ Described:  Provided interface.  Required interfaces.  protocols for valid call sequences.  Requirements for the specified provided services. b) An implemented component: ◦ Realizes a component specification :  Providing the functionality specified by the provided interfaces. Performance evaluation of component-based software systems10

11. 2. Component life-cycle(cont`d)  Information on how the provided services of the implementation, call the required interfaces.  Information about behavior, resource demands of the component,  Input parameters. c) A deployed component: ◦ Results from assembling:  Implemented component to other components.  Allocating it onto a hardware node. Performance evaluation of component-based software systems11

12. ◦ component consists of:  Implementation information.  Deployment information. ◦ Information about :  Which provided services can actually be offered.  Component container.  Operating system.  Hardware. Performance evaluation of component-based software systems12 2. Component life-cycle(cont`d)

13. d)A runtime component: ◦ May serve client requests:  Object in memory. ◦ Components have an internal state. ◦ Check the violation of valid protocol states. ◦ Workload (i.e. the number of clients calling the component). ◦ Input parameters. ◦ Information about concurrently running processes. Performance evaluation of component-based software systems13

14. Feature diagram for component performance model Fig.2, Component performance models[1] Performance evaluation of component-based software systems14 component performance model Schedulable resource demands Limited resource demands Control flow Required service calls Parameter dependencies Internal state

15. 3. Requirements for a component performance modeling language. a) Schedulable resource demands: ◦ Access different active resources:  Processor or storage device. ◦ Finding bottlenecks: ◦ Necessary to scheduling:  Resource demands. ◦ The unit of scheduled resource demands:  Platform-dependent timing value (seconds).  Platform-independent value (CPU cycles). Performance evaluation of component-based software systems15

16. b) Limited resource demands: ◦ Semaphores, memory buffers.  Waiting delays for contention with other concurrently executed services. c) Control flow: ◦ Changing the resource contention:  The order of accessing resources.  Calling required services. Performance evaluation of component-based software systems16 3. Requirements for acomponent performancemodeling language. (cont`d)

17. d) Required service calls: ◦ Make calls to required services. ◦ Service calls:  Synchronous (the caller blocks until receiving an answer).  Asynchronous (the caller continues execution immediately after the call). Performance evaluation of component-based software systems17 3. Requirements for acomponent performancemodeling language. (cont`d)

18. e) Parameter dependencies: ◦ Values of service parameters change:  Execution time.  Memory consumption.  Its accesses to active or passive resources.  Number of calls to required services. f) Internal state: ◦ Global state (equal for all clients). ◦ Local state (different for each client). Performance evaluation of component-based software systems18 3. Requirements for acomponent performancemodeling language(cont`d)

19. Performance evaluation methods Performance evaluation of component-based software systems19 Fig3. Overview of Performance evaluation method[1] Performance evaluation Approaches Component-base Software Systems Main ApproachSupplemental Approach Prediction Approaches based on UML Monitoring Approaches for Iimplementation CB-SPRef Cam

20. Main approaches: ◦ Provide full performance evaluation processes. Supplemental approaches: ◦ Focus on specific aspects,:  Measuring individual components. Performance evaluation of component-based software systems20 Performance evaluation methods

21. Main approaches Prediction approaches based on UML: This group: ◦ Component-based software systems modeled with the unified modeling language (UML). ◦ Approaches in this group target performance predictions during design time. Modeled with the Unified Modelling Language (UML) ◦ UML modeling component behavior with:  Sequence, activity, and collaboration diagrams. ◦ Component allocation described:  Deployment diagrams. ◦ UML supports:  Modeling performance attributes:  Timing values, workload parameters. Performance evaluation of component-based software systems21

22. The Component-Based Software Performance Engineering(CB-SPE) UML as design model. The CB-SPE framework includes : ◦ Freely available modeling tools (ArgoUML). ◦ Transformation tool to map the UML model to execution graphs. Performance evaluation of component-based software systems22

23. Measurement approaches for component implementations: The goal in measurement approaches : ◦ To derive parameterized performance specifications via multiple measurements. These approaches include: ◦ A testbed to execute the components. RefCAM: ◦ Approach to determine resource demand for software components ◦ Dependency to:  Input parameter value, execution environments. RefCAM: ◦ Records CPU demands. ◦ Applies function fitting techniques for different execution environments. ◦ The results are stored in a repository. Performance evaluation of component-based software systems23 Supplemental approaches

24. Evaluation General features: (CB-SPE) ◦ Target domain.  Distributed system. ◦ Component description language.  UML. ◦ Tool Support:  Modeling, analysis(CB-SPE Tool suite) Performance evaluation of component-based software systems24

25. Related works During the last ten years: ◦ Researchers have proposed many approaches for evaluating the performance of component-based software systems. Approaches deal with both: ◦ Performance prediction. ◦ Performance measurement. Classical performance models: ◦ Queuing networks, stochastic Petri nets, stochastic process algebras. Balsamo reviewed model-based performance prediction methods for general systems. Becker provided an overview of component-based performance modeling and measurements methods. Woodside designed a roadmap for future research in the domain of software performance engineering and recommended to exploit techniques from Model- Driven Development for performance evaluation of component-based systems. Performance evaluation of component-based software systems25

26. Future directions There is limited consensus about the performance modeling language for component-based systems. Component performance can be modeled on different abstraction levels. The question is: ◦ which detail to include into the performance models because of its impact on timing. ◦ which detail to abstract because of its limited impact. The goal is to create an abstraction of a software component: ◦ Allows accurate performance prediction results. Most existing methods do not support modeling internal state and parameter dependencies well. More research and experiments into this direction are necessary. Performance evaluation of component-based software systems26

27. Conclusions Practitioners gain: ◦ An overview of performance evaluation methods proposed in the research. Select methods according to their specific situation. A generic approach applicable on all kinds of component-based systems may not be achievable. Performance evaluation of component-based software systems27

28. Reference H. Koziolek, Performance evaluation of component-based software systems: A survey, Performance Evaluation (2009), doi:10.1016/j.peva.2009.07.007 28