1. Inspections on Testing Aspect-Oriented Programs Rodrigo Manuel Lopes de Matos Moreira 4 th Doctoral Symposium on Informatics Engineering Porto, 6 th February 2009

2. DSIE’09 – 6 th February 2009 Rodrigo Moreira 2 Inspections on Testing Aspect-Oriented Programs Agenda  Introduction  Aspect-Oriented Programming  AOP Quality Issues  AOP Testing Issues  Improving Software Quality Assurance in AOP  Conclusions and Future Work Inspections on Testing AOP Programs > Agenda

3. DSIE’09 – 6 th February 2009 Rodrigo Moreira 3 Inspections on Testing Aspect-Oriented Programs Introduction  AOP is an emerging methodology that best suites the concept of separation of concerns (SOC)  SOC is one of the key principles in Software Engineering. It refers to the ability to identify, encapsulate, and manipulate concrete portions of software that are relevant to a particular concern.  In practice, a software system corresponds to the realization of a set of concerns  Aspect-Oriented Programming (AOP) is a recent programming paradigm that aims at enhancing modularity and thus solving the problem of crosscutting concerns by capturing them into new units of modularity called aspects. Inspections on Testing AOP Programs > Introduction

4. DSIE’09 – 6 th February 2009 Rodrigo Moreira 4 Inspections on Testing Aspect-Oriented Programs Aspect-Oriented Programming  Despite the fact that Object-Oriented programming (OOP) is the most used methodology to manage core concerns, it doesn’t allow decomposition of a problem into all of its concerns  AOP should be applied on systems that have concerns scattered and tangled through the modules Inspections on Testing AOP Programs > Aspect-Oriented Programming  The main goal for AOP is to handle crosscutting concerns

5. DSIE’09 – 6 th February 2009 Rodrigo Moreira 5 Inspections on Testing Aspect-Oriented Programs Concepts Inspections on Testing AOP Programs > AOP > Concepts  Join points  Pointcuts  Advices  Aspects Illustration from “The Case for Aspect Oriented Programming” by André Restivo, 2005

6. DSIE’09 – 6 th February 2009 Rodrigo Moreira 6 Inspections on Testing Aspect-Oriented Programs AOP Quality Issues  The incorrect and undisciplined use of AOP methodology, can lead into several quality issues  When new code blocks are woven into the target application, several risks arise  Since the target application will have its behavior changed, it can cause an impact on software qualities (reliability, functionality, performance and efficiency)  Software correctness might be affected, causing redundancy in the system  AOP also has impact on the architecture of a software system. If it is applied correctly, it can simplify the maintenance and expansibility of the system. Otherwise, it can affect system’s performance, correctness and reliability. Inspections on Testing AOP Programs > AOP Quality Issues

7. DSIE’09 – 6 th February 2009 Rodrigo Moreira 7 Inspections on Testing Aspect-Oriented Programs Quality Improvement  Enable releasing resources and thus allowing developers to focus on the core implementation  Due to better modularization, AOP improves the implementation’s comprehensibility and simplifies the process to integrate new requirements and to accommodate changes  Software quality is able to improve since AOP provides :  Clear responsibilities for individual modules  Consistent implementation  Improved reusability  Improved skill transfer Inspections on Testing AOP Programs > AOP Quality Issues > Quality Improvement

8. DSIE’09 – 6 th February 2009 Rodrigo Moreira 8 Inspections on Testing Aspect-Oriented Programs AOP Testing Issues  AOP is an emerging programming paradigm and due to its evolution, the difficulty of testing AOP is receiving more attention  Likewise, when OO was first introduced as a new programming paradigm, it brought a unique set of both benefits and challenges  AOP testing issues can be categorized as:  AOP characteristics  Aspects conflicts and interferences  Problems with pointcuts  Undisclosed type of errors / bug patterns  Recurring / Symptomatic issues Inspections on Testing AOP Programs > AOP Testing Issues

9. DSIE’09 – 6 th February 2009 Rodrigo Moreira 9 Inspections on Testing Aspect-Oriented Programs AOP Characteristics  AOP has different characteristics than OO, which can pose new issues and therefore new challenges regarding testing :  Aspects do not have independent identity or existence  Aspect implementation can be tightly coupled to their woven context  Control and data dependencies are not readily apparent from the source code of aspects or classes  Emergent behavior  These issues are challenges that cannot be addressed using traditional unit or integration testing techniques  The unique features of AOP don’t manifest in OO or even in procedural programming. Each feature can introduce new fault types that may lead to failures. Inspections on Testing AOP Programs > AOP Testing Issues > AOP Characteristics

10. DSIE’09 – 6 th February 2009 Rodrigo Moreira 10 Inspections on Testing Aspect-Oriented Programs Aspects conflicts and interferences  The aiming of AOP towards obliviousness has been harder to attain, as conflicts between aspects may arise.  When new behavior is added into the system by an aspect, previous tests can be broken  To detect such conflicts but also to understand their interactions (between aspects and base classes)  It is necessary to understand how new added behavior can cause conflicts and how to detect its source Inspections on Testing AOP Programs > AOP Testing Issues > Aspects conflicts and interferences

11. DSIE’09 – 6 th February 2009 Rodrigo Moreira 11 Inspections on Testing Aspect-Oriented Programs Problems with pointcuts  Pointcuts contain specifications that match the join points of a particular type according to a signature that includes a pattern  When dealing with large and complex programs, developers may create pointcuts that may not be correct or might fail its intended purpose  Developers might write pointcut expressions with incorrect strength, originating two scenarios:  if the pattern is too strong some required join points will not be selected  if the pattern is too weak then additional join points will be selected that should be ignored  Testing the strength of pointcuts needs to be addressed with the aim of validating the correctness in their expressions Inspections on Testing AOP Programs > AOP Testing Issues > Problems with pointcuts

12. DSIE’09 – 6 th February 2009 Rodrigo Moreira 12 Inspections on Testing Aspect-Oriented Programs Undisclosed type of errors / bug patterns  Due to the complexity introduced by AOP, such as behavior modification, it turns out to be difficult to detect and locate errors in aspect-oriented programs.  Some bugs may be hidden on either aspects or/and in base classes  Bugs may appear in a program woven by a weaver even if bugs do not exist in individual aspects or objects. These kinds of bugs are caused by weaving policies such as the order weaving followed.  The distinctiveness nature of AOP, leads testers to not have sufficient knowledge on how to write efficient and proper tests, in order to cover most common errors that appear in aspect-oriented programs Inspections on Testing AOP Programs > AOP Testing Issues > Undisclosed type of errors / bug patterns

13. DSIE’09 – 6 th February 2009 Rodrigo Moreira 13 Inspections on Testing Aspect-Oriented Programs Recurring / Symptomatic issues  AOP is being used for a wide range of uses (tracing, runtime assertion and logging). Such diversity enhances additional difficulty on the task to identify aspects and build them for testing as test oracles  Current specification-based tests generation methods are only able to test program behavior specified by terms of invariants  Changing the execution order of statements might also defy some assumptions required for a given statement to work properly  How to determine which code portions can be or should be retested when the aspects are added into the system ?  When a fault occurs, is it related to the aspects or it has to do with the base classes ? Inspections on Testing AOP Programs > AOP Testing Issues > Recurring / Symptomatic issues

14. DSIE’09 – 6 th February 2009 Rodrigo Moreira 14 Inspections on Testing Aspect-Oriented Programs Improving Software Quality Assurance in AOP  A great deal of research for testing aspect-oriented programs has been conducted over the past years  Suggestions aiming to improve software quality in AOP :  Fault Model  Bug Patterns  Unit Testing  Mutation Testing  Test Oracles  State Based Incremental Testing Inspections on Testing AOP Programs > Improving Software Quality Assurance in AOP

15. DSIE’09 – 6 th February 2009 Rodrigo Moreira 15 Inspections on Testing Aspect-Oriented Programs Fault Model  Reflects the structural and behavioral characteristics of aspect-oriented programs  Aims to take an effective step towards systematic testing of aspect-oriented programs  Alexander and Bieman proposed a candidate fault model for AOP :  Incorrect strength in pointcut patterns  Incorrect aspect precedence  Failure to establish postconditions  Failure to preserve state invariants  Incorrect focus of control flow  Incorrect changes in control dependencies Inspections on Testing AOP Programs > Improving Software Quality Assurance in AOP > Fault Model

16. DSIE’09 – 6 th February 2009 Rodrigo Moreira 16 Inspections on Testing Aspect-Oriented Programs Bug Patterns  Referred as erroneous code (can be due to bad programming practices) that constantly persists failing  Zhang and Zhao, identified 6 bug patterns in AspectJ:  Infinite loop – accidental matching of unexpected join points  Scope of the advice – if a parameter is reassigned it will have no changes outside of the advice  Multiple advice invocation – execution sequence might affect the result of a program  Unmatched join point – due to args() primitive  Misuse of getTarget() – due to AspectJ reflection mechanism  Introduction interference – affects class runtime behaviors unexpectedly Inspections on Testing AOP Programs > Improving Software Quality Assurance in AOP > Bug Patterns

17. DSIE’09 – 6 th February 2009 Rodrigo Moreira 17 Inspections on Testing Aspect-Oriented Programs Unit Testing  Unit testing can be used to detect conflicts between aspects  A. Restivo and A. Aguiar, addressed this challenge, by means of a methodology in order to improve management of conflicts and also by a tool to support the methodology  Authors claim there is a need for aspects to be able to perform three main activities :  to announce which aspects they are expected to break;  which aspects they depend on;  and which they are adding to the system Inspections on Testing AOP Programs > Improving Software Quality Assurance in AOP > Unit Testing

18. DSIE’09 – 6 th February 2009 Rodrigo Moreira 18 Inspections on Testing Aspect-Oriented Programs Mutation Testing  In order to test pointcuts for their strength, Anbalagan and Xie proposed to apply mutation testing  Mutation testing is a fault-based technique that can be used to inject faults into a program. Its purpose is to help to locate and expose weaknesses in test suites.  The proposed mutation testing of pointcuts is performed in two steps:  create effective mutants of a pointcut expression  test these mutants using the designed test data  To reduce human efforts in mutation testing of pointcuts – due to a large set of mutants – authors developed a framework to automatically generate relevant mutants Inspections on Testing AOP Programs > Improving Software Quality Assurance in AOP > Mutation Testing

19. DSIE’09 – 6 th February 2009 Rodrigo Moreira 19 Inspections on Testing Aspect-Oriented Programs Test Oracles  Xu and Yang, guided their effort in solving the problem that relies on how to build specific testing aspects which can be identified as test oracles  A new concept is introduced: application-specific test. This concept stands as the separation of concerns on specific application of common aspect-oriented programs’ aspects  To build application-specific aspects for testing, the use of Aspect- Oriented Test Description Language (AOTDL) is required Inspections on Testing AOP Programs > Improving Software Quality Assurance in AOP > Test Oracles

20. DSIE’09 – 6 th February 2009 Rodrigo Moreira 20 Inspections on Testing Aspect-Oriented Programs State Based Incremental Testing  Incremental approach to test if aspect-oriented programs and their base classes conform to their respective behavior models, presented by D. Xu and W. Xu  An aspect-oriented extension is used to state models in order to capture the impact of aspects on state transitions of base class objects  This work demonstrated that for a majority of base class tests can in fact be reused for aspects but some modifications to some of them are required  Authors state that their incremental testing approach is somewhat similar to traditional regression testing with the difference that aspects are feasible to investigate systematic reuse and modification of the existing tests Inspections on Testing AOP Programs > Improving Software Quality Assurance in AOP > State Based Incremental Testing

21. DSIE’09 – 6 th February 2009 Rodrigo Moreira 21 Inspections on Testing Aspect-Oriented Programs Conclusions and Future Work  Most of the proposed approaches to achieve and enhance software quality in AOP do not solve every problem  Existing approaches cannot be seen as a “silver bullet” towards testing aspect- oriented programs  Most of the existing work on testing aspect-oriented programs is being directed only towards one of the existing programming languages for AOP – AspectJ  Some the testing issues identified in this work appear related with AspectJ. However, it is not clear if such issues also apply to other AOP tools such as, PostSharp and Spring.NET  A future work effort could be directed on the innovation on how to apply other testing methodologies such as FIT (Framework for Integrated Test) to AOP.  Another idea refers to the usage of aspects for testing aspects Inspections on Testing AOP Programs > Conclusions and Future Work

22. DSIE’09 – 6 th February 2009 Rodrigo Moreira 22 Inspections on Testing Aspect-Oriented Programs Questions ? Inspections on Testing AOP Programs > Questions