ASTA Aspect Software Testing Assistant Juha Gustafsson, Juha Taina, Jukka Viljamaa University of Helsinki

2/21 Motivation Cross-Cutting Concerns in Software Implementation modules Security Logging Persistence Cross-cutting concerns clutter the main class model

3/21 Logger log(s: String) out: OutputStream

4/21 An Example Money Transfer: Logical Level void transfer(Account from, Account to, int amount) { if (from.getBalance() < amount) { throw new InsufficientFundsException(); } from.withdraw(amount); to.deposit(amount); }

5/21 void transfer(Account from, Account to, int amount) { if (!getCurrentUser().canPerform(OP_TRANSFER)) { throw new SecurityException(); } Transaction tx = database.newTransaction(); if (from.getBalance() < amount) { tx.rollback(); throw new InsufficientFundsException(); } from.withdraw(amount); to.deposit(amount); systemLog.logOperation(OP_TRANSFER, from, to, amount); tx.commit(); } An Example Money Transfer: Implementation Details Security Logging Persistence

6/21 Solution Aspect-Oriented Programming (AOP) Separating cross-cutting concerns from the core concerns (i.e. from the main class model) Aspect = a definition of a cross-cutting concern  pointcut = a pattern that selects join points (i.e. elements that appear in method bodies)  advice = a method-like construct that is executed in the selected join points (before, after, around)  weaving = the process of attaching advice to the appropriate join points

7/21 An example AOP program HelloWorld in the AspectJ language Go to … … and watch the demo!

8/21 A new question arises How to test AOP? Do we test what we intend to test?  How to prioritise our tests?  How to pick a good testing strategy? When should we stop testing? In AOP these are especially difficult questions because  the execution context of an aspect depends on the (internal implementation of the) class it is woven to  relating a failure to the fault that causes it is difficult the control and data flows are not explicit a particular aspect weave order can cause problems

9/21 A new question arises How to test AOP? Visualising AOP testing is not straightforward  what kinds of test coverage criteria to use and visualise (is it sufficient to show that we have visited an aspect once, how to show the weaving context)?  current control flow graphs are insufficient to visualise the test execution paths in the presence of aspects  high-level coverage visualisation for AOP testing is a completely open question Yet, visualisation of AOP testing is (at least) as important as visualisation of traditional (OO) testing

10/21 Testing AOP Fault Sources Fault in the core concern code Fault in the (context independent) aspect code Fault created by interactions between the aspect and the core concern code Fault created when more than one aspect is woven into the core concern code

11/21 Testing AOP [Alexander, Bieman, Andrews, 2004] Fault Model & Additional Testing Required Incorrect strength in pointcut patterns  test the aspect Incorrect aspect precedence  test all weave orders Incorrect focus of control flow  use a form of condition coverage of pointcut designators Failure to establish expected postconditions  use the original test set to re-test all methods that have the aspect Failure to preserve state invariants  same as in previous Incorrect changes in control dependencies  same as in previous

12/21 AOP Testing Theory Coverage criteria Coupling metrics Control flow formalism …

13/21 Tool Support Tool support is essential for systematic testing Not many tools exist for AOP testing  coverage tools  visualisation tools  scripting languages & environments to automate AOP testing

14/21 Research Background Due to project RITA (fRamework Intgration and Testing Application) we have a strong background in white-box test execution and analysis RITA is intended for product family testing, but similar ideas and issues arise in AOP testing RITA as a prototype is not suitable for large- scale testing but its ideas are relevant

15/21 RITA Functionality Rita supports the following:  Java program testing  Product family or single application testing  JUnit and private test execution  Full support for data flow diagrams  Visualisation and white-box coverages on four levels: Method level: traditional coverages Object level: object and class coverages Package level: object coupling, reference coverages Application level: multiple application coverages

16/21 Lessons learned from RITA A White-Box Testing Software (WBTS) must support traditional code  RITA: not all applications are in product families  ASPECTS: not all applications use aspects WBTS should be integrated to a software development environment  While stand-alone software can have similar functionality than integrated software, usability issues favor the latter approach Code coverage is not sufficient in WBTS  The higher is the abstraction of testing, the more complex coverages are needed.  Even at code level code coverage gives weak results.

17/21 Aspect-Oriented Testing Tool Based on our experiences in RITA we propose the following requirements for an Aspect- Oriented Testing Tool (AOTT):  It has to support testing of traditional software  It must be integrated to a well-known and actively updated programming environment such as Eclipse  It has to support several levels of white-box testing including program-level, package-level, object-level, aspect-level, and method-level Aspect-level: testing and visualisation are inside a single aspect.

18/21 AOTT Requirements Continue More AOTT requirements:  It has to calculate static mesurements of the software: Lines of code (LOC) Aspects of code (AOC) Coupling between objects Depth of inheritance tree Depth of aspect tree (aspects in aspects) Coupling between code and aspects etc.  It has to support integration between dynamic test execution results and static measurements How? This is an open research question  Something else? Definitely! This is a work-in-progress list

19/21 Research directions [Alexander, Bieman, Andrews, 2004] Open Questions Can we measure test coverage after weaving?  also visualisation How do we test aspects that interact with the core concern code?  coupling between an aspect and core concern code as a measure of testability of the aspect How do we test aspects that interfere with each other?  inter-aspect coupling How do we test aspects whose effects must span more than one concern? Are there ways to test aspects on their own?  probably not… ? (Can we reverse engineer the weave process?)  how does this relate to testing

20/21 Validation How to validate that  Theory is correct?  Application is effecient? Alternatives:  Formalism  Empirical controlled experiments  Practical experiments in industrial settings All alternatives are  Relevant  Needed.

21/21 Summary To be written...