imbus AG Kleinseebacher Strasse 9 D Moehrendorf Germany Phone: +49 (0) Fax: +49 (0) Web: © imbus AG 2001 Automated Generation and Execution of Test Suites Bernd Mattern imbus AG Kleinseebacher Strasse Moehrendorf The Project

© imbus AG, Bernd MatternThe AGEDIS Project 2 AGEDIS What is AGEDIS? A utomated G eneration and E xecution of Test Suites for Dis tributed Component-based Software Information Society Technologies (IST) Programme Total budget: 4,3 M EUREC pays: 2,3 M EUR

© imbus AG, Bernd MatternThe AGEDIS Project 3 AGEDIS Agenda 1. Motivation 2. Goals of AGEDIS 3. Overview of AGEDIS 4. AGEDIS Modelling Language 5. Other Approaches 6. Summary and Outlook

© imbus AG, Bernd MatternThe AGEDIS Project 4 AGEDIS 1. Motivation Design Specification Review Functional Specification Test Specification

© imbus AG, Bernd MatternThe AGEDIS Project 5 AGEDIS 1. Motivation Design Specification Review Functional Specification Test Specification AGEDIS Model automatic derivation

© imbus AG, Bernd MatternThe AGEDIS Project 6 AGEDIS 2. Goals of AGEDIS Creation of a new testing methodology Cheaper software development costs Continuous approach to testing from modelling the software to the evaluation of the test run results Adoption of standards

© imbus AG, Bernd MatternThe AGEDIS Project 7 AGEDIS 3. Partners (Institutions) IBM Haifa, Israel: Alan Hartman, Kenneth Nagin, Sergey Olvovsky, Andrei Kirshin,... Oxford University, UK: Jim Davies, Alessandra Cavarra France Telecom R&D, France: Yves-Marie Quemener Intrasoft International, Luxembourg, Greece: Kyriakos Bechrakis, Antonis Ramfos

© imbus AG, Bernd MatternThe AGEDIS Project 8 AGEDIS IBM Hursley, UK: Ian Craggs VERIMAG - Université Joseph Fourier, France: Laurent Mounier, Yassine Lakhnech Irisa, France: Thierry Jeron 3. Partners (Institutions) imbus AG, Germany: Manuela Heigl, Johannes Trost, Bernd Nossem, Klaudia Dussa-Zieger,...

© imbus AG, Bernd MatternThe AGEDIS Project 9 AGEDIS User Interface Software Model Intermediate Format Encoding Generator Abstract Test Suite Execution Engine Suite Execution Trace AnalyzerVisualizer Test Goals Compiler Test Execution Directives 3. Architecture

© imbus AG, Bernd MatternThe AGEDIS Project 10 AGEDIS 3. Work Package Structure Work Package (WP)total effort WP1 - Project Management33 PM WP2 - Software Modelling Language25 PM WP3 - Modelling Language Compiler31 PM WP4 - Test Generation Tool32 PM WP5 - Test Execution Tool32 PM WP6 - Test Productivity Tool37 PM WP7 - Test Automation Environment36 PM WP8 - Application Experiments56 PM WP9 - Technology Transfer54 PM Sum336 PM (28 PY)

© imbus AG, Bernd MatternThe AGEDIS Project 11 AGEDIS 3. Roles of Partners Coordinator, leader of WP3, WP5, WP6, moderator exp 2 Leader WP2 and educational expertise tool user, delivers test subjects tool user, leader of WP7, integration, delivers test subjects tool user, supplier of test subjects leader WP4, moderator exp 1 tool user, testing expertise, leader of WP9, moderator exp 3, 4, 5

© imbus AG, Bernd MatternThe AGEDIS Project 12 AGEDIS Survey Language Spec First Experiments Prototypes: Compiler Generator Executor 2nd Experiments Final Tools Exploitation Publication 4 months 12 months requirements design implementation test 12 months 3 months Today 3. Flow of Events

© imbus AG, Bernd MatternThe AGEDIS Project 13 AGEDIS UML based Subset of UML diagrams Definition of special semantics Definition of software design Definition of test goals / test directives 4. Modelling Language

© imbus AG, Bernd MatternThe AGEDIS Project 14 AGEDIS 4. From Software Design Class Diagrams: Class1 +attr1 +attr2 +operation1 +operation2 Class2 -attr3 #attr4 -operation1 +operation2 Class3 -attr3 -attr4 #operation1 +operation2 Class4 #attr1 #attr2 +operation1 #operation2 0..*

© imbus AG, Bernd MatternThe AGEDIS Project 15 AGEDIS 4. From Software Design Class1 +attr1 +attr2 +operation1 +operation2 state 1state 3state 4state 2 every state has: -entry & exit actions -internal transitions -NEW in AGEDIS: deferred events timer events decision final initial trigger[gard]/action For each class: State transition diagrams:

© imbus AG, Bernd MatternThe AGEDIS Project 16 AGEDIS 4. AGEDIS Semantic extension > Signal1 {controllable, observable} +attr1 +attr2 > Signal2 {controllable} +attr1 +attr2 > Signal3 {observable} +attr1 +attr2 > Signal4 +attr1 +attr2 Controllable: Signals, that can be sent to the system by the tester Observable: Signals from the system, that can be validated by the tester Stereotype > used to specify events and actions in state diagrams:

© imbus AG, Bernd MatternThe AGEDIS Project 17 AGEDIS Object Diagrams: - Define states of the system during test. Attributes are defined. - AGEDIS Semantic extension: Special stereotypes are given to diagrams: >, >, >, > > 4. Test Directives obj1:Class2 -attr3=10 #attr4=0 obj2:Class3 -attr3=Hi -attr4=Bye

© imbus AG, Bernd MatternThe AGEDIS Project 18 AGEDIS > 4. Test Directives What does the AGEDIS test generator do? Searches for test cases: Good case!

© imbus AG, Bernd MatternThe AGEDIS Project 19 AGEDIS > 4. Test Directives another Good case! What does the AGEDIS test generator do? Searches for test cases:

© imbus AG, Bernd MatternThe AGEDIS Project 20 AGEDIS > 4. Test Directives Bad cases! What does the AGEDIS test generator do? Searches for test cases:

© imbus AG, Bernd MatternThe AGEDIS Project 21 AGEDIS 4. Test Directives state 1RejectAcceptstate 2 Alternative approach: -Definition of a system-wide state transition machine -AGEDIS semantic extension: Accept and Reject state signal1 signal2 signal1

© imbus AG, Bernd MatternThe AGEDIS Project 22 AGEDIS Lots of sequences including: Variations in work flows Variations in input data Variations in output data Summed up: An abstract test suite Output format: XML Details under discussion 4. Output from AGEDIS obj1:Class1obj2:Class2env:Env get(1):Signal3 put(1):Signal1 put(0):Signal2 get(0):Signal3 send(1):Signal4 receive(1):Signal4

© imbus AG, Bernd MatternThe AGEDIS Project 23 AGEDIS Testbeans GTC (Gotcha - TCBeans from IBM) Meta-Words (WinRunner Script based method from imbus) Your collection of black or white box test functions 4. Bringing the test suite to life

© imbus AG, Bernd MatternThe AGEDIS Project 24 AGEDIS Arbitrary GUI Test tools e.g. WinRunner (Mercury Interactive) 4. Test Execution Engine Protocol interface driver customer tools

© imbus AG, Bernd MatternThe AGEDIS Project 25 AGEDIS StP/UML (Software through pictures, Aonix) template technology > 50% source code generation j-UML (Extension for Java, VTT, Finland) Symbols for special Java features no tool xUML (Kennedy Carter) complete Action Language for source code generation Tool bases validation of design through test cases 5. Other approaches

© imbus AG, Bernd MatternThe AGEDIS Project 26 AGEDIS AGEDIS will reduce test specification and review effort The AGEDIS software derives test cases from UML diagram via test directives AGEDIS brings test documentation closer to development departments AGEDIS makes testing more complete AGEDIS makes testing more fun 6. Conclusion

© imbus AG, Bernd MatternThe AGEDIS Project 27 AGEDIS Contact Headquarter Erlangen/Moehrendorf: imbus AG Tel:09131/ Kleinseebacher Strasse 9 Fax:09131/ Moehrendorf Germany Office Munich: imbus AG Tel:089/ Unter der Linde 16 Fax:089/ München Germany Office Frankfurt: imbus Rhein-Main GmbHTel:06192/ Zanggasse 6 Fax:06192/ Hofheim Germany

© imbus AG, Bernd MatternThe AGEDIS Project 28 AGEDIS

© imbus AG, Bernd MatternThe AGEDIS Project 29 AGEDIS Download further information

© imbus AG, Bernd MatternThe AGEDIS Project 30 AGEDIS Discussion and Newsletter