Survey of Tools to Support Safe Adaptation with Validation Alain Esteva-Ramirez School of Computing and Information Sciences Florida International University Bárbara Morales-Quiñones Department of Computer Engineering University of Puerto Rico-Mayaguez 06/26/2007 REU Summer Program

2 Outline  Introduction  Testing Autonomic Systems  Motivation  Background  Safe Adaptation by Zhang et. al  Tool Classification & Criteria  Survey of Tools  Selection of Tools  Questions?

3 Introduction:  Autonomic Computing  Automated low level task/actions  Specify behavior as high level policies  Self-management features  Testing Autonomic Systems  Requires testing prior to initial deployment  Requires runtime testing, since structure and behavior can change at runtime  Pioneers of autonomic computing stated that validation is one of the grand challenges of autonomic computing. Testing Autonomic Systems (1)

4 Introduction:  Two approaches developed by King et al. [1]  Replication with Validation  Only feasible when managed resources can be replicated  Requires the system to create and/or maintain copies of the managed resource for validation purposes  Changes implemented and validated on copies  Safe Adaptation with Validation  Validates changes resulting from self-management as part of a safe adaptation process  It can be used when duplicating managed resources are too expensive, impractical, or impossible  Occurs directly on the managed resource, during execution. [1] Towards Self-Testing in Autonomic Computing Systems Testing Autonomic Systems (2)

5 Motivation  Survey represents preliminary work for Testing Autonomic Computing Systems During Safe Adaptation.  Motivation stems from the need to test autonomic computing systems at runtime (i.e., to avoid the high cost of system failures.)  Since the strategy is based on safe adaptation, investigation of tools can be useful for building dependable adaptive (and autonomic) systems.  Many new tools/plugins have emerged; integrated development platforms, open-source comm.

6 Background  Safe adaptation  Developed by Zhang et al. (WADS 2004)  Directed towards using a disciplined approach to building adaptive systems.  An adaptation is safe if and only if:  It does not violate the dependencies between components  It does not interrupt any critical communications that could result in erroneous conditions [2] Enabling Safe Dynamic Component-Based Software Adaptation Safe Adaptation by Zhang et al. (1)

7 Background Safe Adaptation by Zhang et al. (2) Source: J. Zhang, B. H. C. Cheng, Z. Yang, and P. K. McKinley. Enabling safe dynamic component-based software adaptation. In WADS, pages 194–211, 2004.

8 Tool Classification  Dependency Analysis Tools  Partially automate safe adaptation process.  Extract dependency relationships among and between components.  Metrics Tools  Allows to measure efficiency based on certain performance metrics (memory, response time). Aids validation of self-optimization features.  Complexity metrics – automatically generate costs of adaptation steps.  Unit Testing Tools  Support or enhance previous unit testing (REU 2006). [2] Enabling Safe Dynamic Component-Based Software Adaptation

9 Tool Selection Criteria (1)  Dependency Analysis  Exportable dependencies – we need a way to access the generated information, to later analyze it.  Graphical visualization – aids us in visualizing these dependencies  Dependency cycle detection – it is important to know these to accurately manage the impacts of changes

10 Tool Selection Criteria (2)  Performance Analysis  Load – mechanisms to check object’s memory usage.  Speed – mechanisms to evaluate response time of method calls.  Complexity – code complexity related metrics like cyclomatic complexity and coupling between objects

 Unit Testing Support  Java Unit Testing – support or enhance unit testing  Code Coverage – support dynamic analysis of code for unit test coverage.  Branch and line coverage Tool Selection Criteria (3)

12 Survey of Tools  JDepend  Eclipse plugin that exports generated design metrics.  Dependency Finder  Independent tool  Filters dependencies by packages, classes or features  Code Analysis Plugin  Information is presented through diagrams that help visualize the dependencies. Dependency Analysis Tools

13 Survey of Tools  Test and Performance Tools Platform ( TPTP)  Provides framework in which developer build test and performance tools that integrates with Eclipse  Finds problems faster and with less difficulty  Finds performance bottlenecks and other metrics easily  Addresses the entire test and performance cycle including test editing and execution, monitoring, profiling, among other capabilities. Performance Metrics Tools

14 Survey of Tools  JUnitPerf  JUnit test decorators that are used to measure efficiency of JUnit tests.  Timed tests – create a top level bound  Load tests – creates an artificial load  Cobertura  Provides XML coverage reports from the system level to an individual line of code.  Provides complexity metrics.  Provides a maxmemory attribute that helps to restrict the amount of memory used. Unit Test Support Tools

15 Tool Selection  Dependency Analysis  JDepend – exporting XML dependency report  CAP – Visualizing dependency graphs  Performance Analysis  TPTP – addresses the entire test and performance life cycle  Unit Testing  JUnitPerf – measure efficiency of JUnit tests  Cobertura – exporting XML coverage report

16 References  [1] Towards Self-Testing in Autonomic Computing Systems  [2] Enabling Safe Dynamic Component- Based Software Adaptation  [3] Survey of Tools to Supporting Testing Autonomic Computing Systems During Safe Adaptation

17 Questions? Questions, comments and queries.