ETICS All Hands meeting B ologna, October , 2006 WP4 Status Eva TAKACS

2 Overview Objectives of WP4Objectives of WP4 Events of WP4Events of WP4 Activities of WP4Activities of WP4 Deliverable listDeliverable list Present statusPresent status Future workFuture work Discussion pointsDiscussion points

3 Objectives of WP4 The objectives of WP4 are: to collect requirements for unit, functional and regression testing to deploy, adapt and maintain the software testing tools in addition to collect and adapt the related testing methodology to identify quality metrics, and evaluate the different methods and tools based on them, provide measurement collectors for the reporting engine

4 Deliverable list of WP4 D4.1 Requirements and specifications for unit, functional and regression testing – –(PM06 4D Soft) D4.2 Distributed test execution system (prototype) – –(PM12 4D Soft) D4.3 Distributed test execution system (final release) Coherent documentation of the selected software testing tools and methodology – –(PM22 UoW)

5 Work has been done a comprehensive study of the relevant testing methods has been donea comprehensive study of the relevant testing methods has been done a selected description of well-known or state-of-art testing tools has been done. Distributed environment was taken into account.a selected description of well-known or state-of-art testing tools has been done. Distributed environment was taken into account. based on the existing methods and tools requirements for both the testing methods and tools has been donebased on the existing methods and tools requirements for both the testing methods and tools has been done Output: Output: D4.1 Requirements and specifications for unit, functional and regression testing

6 Events of WP4 WP4 Internal meetingWP4 Internal meeting –1st october 2006 at CERN Geneva Scope of the meetingScope of the meeting –to discuss on detail the requirements relating testing tools and methods –connections with other WPs and their tasks Outcome of the meetingOutcome of the meeting –Central part of the testing toolkit to be integrated in ETICS should be a Test and Metrics Plug-in Framework and the testing tools and metrics calculation utilities will be plugged here

7 Why a plugin framework? Making an application extensible using the plugin system has a strong advantage over other techniques: Making an application extensible using the plugin system has a strong advantage over other techniques: It allows third-party developers to easily enhance software in a way that is loosely coupled: only the plugin API is required to remain stable.It allows third-party developers to easily enhance software in a way that is loosely coupled: only the plugin API is required to remain stable.

8 Why a Plugin Framework in ETICS? The rationale behind using „Eclipse style” plug-in architecture is that most of present and/or future ETICS users will come with some tests and testing tools already in place and of course they would like to use them work with ETICS that most of present and/or future ETICS users will come with some tests and testing tools already in place and of course they would like to use them work with ETICS For example: A specific user would like to use his charting system when creating reports after running some functional testcases with TestManager

9 Why a Plugin Framework in ETICS? On the other hand they should be provided with „built-in” (default) testing tools when they do not have their own but would like to use something (Assuming – we plugged the best tools beforehand) For example: Users would like to use a coverage tool to measure the effectiveness of their Junit testcases but they do not have one at hand. Then he can use EMMA provided by the framework.

10 Activities of WP4 Two main areas: NMI – as a testing tool Test and Metrics Plugin Framework – –Architecture of the framework – –Test tools to be included as plugins – –Test tools and methodologies cross reference documentation – –Metrics

11 Plugin Framework – First steps It was decided that as a first step –TestManager a testdriver developed in CERN and used for gLite functional testing –Two coverage tools –EMMA, an open source tool requested by future ETICS users –JavaCov, a commercial tool providing a very strong MC/DC coverage criterion will be taken into account.

12 Next steps Testing coverage tools for ETICS code (gLite) in a real environment Having TestManager in a modularised way Architectural specification and prototype for the Plug-in Framework

13 Future work NMI- as a testing tool – prototypeNMI- as a testing tool – prototype Test and Metrics Plug-in Framework – prototype, architectural specificationTest and Metrics Plug-in Framework – prototype, architectural specification NMI- as a testing tool – final release, documentationNMI- as a testing tool – final release, documentation Test and Metrics Plug-in Framework – final release, documentationTest and Metrics Plug-in Framework – final release, documentation

14 Other topics Some of the user requirements:Some of the user requirements: Dependency analysisDependency analysis For example: in case of gLite services helping the testers to identify the „calling chain” of services Workflow system for designing system level testsWorkflow system for designing system level tests