ETICS All Hands meeting B ologna, October , 2006 WP4 Test and Metrics Plugin Framework (WP4) (WP4) Eva TAKACS

2 Overview Why a plugin frameworkWhy a plugin framework Advantages of a Plugin Framework in generalAdvantages of a Plugin Framework in general Core functionality and first steps of the Test and Metrics Plugin FrameworkCore functionality and first steps of the Test and Metrics Plugin Framework Basic architecture of a plugin framework in generalBasic architecture of a plugin framework in general Java Plugin Framework – implementation of a plugin architectureJava Plugin Framework – implementation of a plugin architecture User scenariosUser scenarios Basic architecture of the Test and Metrics Plugin FrameworkBasic architecture of the Test and Metrics Plugin Framework ConclusionConclusion

3 Advantages of a plugin framework in general Making an application extensible using the plugin system has strong advantages over other techniques: Making an application extensible using the plugin system has strong advantages over other techniques: It allows third-party developers to easily enhance software in a way that is loosely coupled: only the plugin API is required to remain stable.It allows third-party developers to easily enhance software in a way that is loosely coupled: only the plugin API is required to remain stable. It allows new plugins to be discovered flexibly.It allows new plugins to be discovered flexibly. More precisely, More precisely, –plugins can be loaded and saved when a program is first run, or re-discovered each time the program starts up –allows the discovery of new plugins installed after the program has started.

4 Why a Plugin Framework for ETICS? The rational behind using „Eclipse style” plug-in architecture is that most of present and/or future ETICS users will with some tests and testing tools already in place and of course they would like to use them when working with ETICS that most of present and/or future ETICS users will with some tests and testing tools already in place and of course they would like to use them when working with ETICS On the other hand they should be provided with „default” testing tools when they do not have their own but would like to use somethingthey should be provided with „default” testing tools when they do not have their own but would like to use something

5 Core functionality of the Test and Metrics Plugin Framework Core functionality of the framework is consisted of Existing tools (default plug-ins)Existing tools (default plug-ins) –TestManager – a testdriver used for gLite testing already in use –Two coverage tools for Java testcases –Emma –JavaCov –other (Cobertura ?) A plugin manager system able to handle the plugins and user profile settings from the web interface and/or command-line interfaceA plugin manager system able to handle the plugins and user profile settings from the web interface and/or command-line interface

6 Plugin Framework – First steps 1 It was decided that as a first step –TestManager a testdriver developed in CERN and used for gLite functional testing –Two coverage tools –EMMA, an open source tool requested by future ETICS users –JavaCov, a commercial tool providing a very strong MC/DC coverage criterion will be taken into account.

7 Plugin Framework – First steps 2 To examine how these tree tools can be integrated in a future plugin environment to „modularise” TestManagerto „modularise” TestManager to have a closer look to the running ways of coverage toolsto have a closer look to the running ways of coverage tools

8 EMMA EMMA – is an open source coverage tool Supported coverage typesSupported coverage types –class –method –line –basic block (branch) Supported Java version: 1.5.*Supported Java version: 1.5.* Running way:Running way: –configurable command line –ANT integration –no Eclipse plugin

9 EMMA - JUnit EMMA – Junit integration EMMA does not support implicitely "JUnit integration„EMMA does not support implicitely "JUnit integration„ and not assume any particular testing methodologyand not assume any particular testing methodology But after doing some configuration can be used for measuring the coverage of Junit test casesBut after doing some configuration can be used for measuring the coverage of Junit test cases

10 JavaCov JavaCovJavaCov Supported coverage typesSupported coverage types –class –method –line –basic block (branch) –MC/DC (refined version of branch, required in safety-critical systems) Supported Java version: 1.4.* ?????????Supported Java version: 1.4.* ????????? Running way:Running way: –ANT integration –Eclipse plugin –no separate command line

11 JavaCov - JUnit JavaCov – JUnit integrationJavaCov – JUnit integration supports implicitely "JUnit integration„ without extra configurationsupports implicitely "JUnit integration„ without extra configuration at the same time does not assume any particular test methodat the same time does not assume any particular test method

12 JavaCov ? Cobertura Because of not supporting java 1.5 and the company is not going to upgrade JavaCov this year it can be replaced by an other coverage tool. One candidate can be Cobertura – an other open source product but without supporting MC/DC criterion.

13 Terms refering to a Plugin Architecture Basic terms: a runtime engine that dynamically discovers and loads "plug-ins„a runtime engine that dynamically discovers and loads "plug-ins„ A plug-in is a structured component that contributes code and resources to the „core” system and describes itself to runtime engine using a "manifest file„ usually an xml.A plug-in is a structured component that contributes code and resources to the „core” system and describes itself to runtime engine using a "manifest file„ usually an xml. Runtime engine maintains a registry of available plug-ins and the functions they provide (via extension points and extensions)Runtime engine maintains a registry of available plug-ins and the functions they provide (via extension points and extensions)registryextension pointsextensionsregistryextension pointsextensions extension points, well-defined methods that can be extended by other plug-ins.extension points, well-defined methods that can be extended by other plug-ins. extension to the system – when an implementatin is added to an extension poitextension to the system – when an implementatin is added to an extension poit

14 Requirements Important requirement of a plugin frameworkImportant requirement of a plugin framework Because of memory and performance issues the installed plug-ins should not be loaded (instantiated) until they are called. They are added to a central registry at application start-up and are loaded when necessary.Because of memory and performance issues the installed plug-ins should not be loaded (instantiated) until they are called. They are added to a central registry at application start-up and are loaded when necessary.

15 Basic Plug-in architecture Plugin Manager - loads and activates plugins - maintains a plug-in registry - finds the physical location of plugins Plugin_1 extension point Plugin_2 Plugin_n API for defining Manifest files Manifest2.xml Manifestn.xml Manifest1.xml Plugin_11

16 Java Plugin Framework JPF (Java Plugin Framework) is a general-purpose open source plug-in framework intended to build extendable Java applications Main features: Open framework architecture but having a „default” implementationOpen framework architecture but having a „default” implementation Lazy plug-in activation (Plug-in classes are loaded into memory only when they are actually needed )Lazy plug-in activation (Plug-in classes are loaded into memory only when they are actually needed ) "On the fly" plug-in registration and activation"On the fly" plug-in registration and activation Xml based manifest files (Eclipse now uses OSGI)Xml based manifest files (Eclipse now uses OSGI)

17 OSGI The OSGi framework specification forms the basis of the Eclipse Runtime. As of Eclipse 3.0, the Runtime is fully based on the OSGi notion of bundle (equivalent to Eclipse plug-ins). The OSGi framework implementation in Eclipse 3.0 is fully compliant with the OSGi framework specification R3.0. The OSGi implementation in Eclipse and the upcoming Eclipse 3.2 fully implement the OSGi R4.0 framework specification.

18 User scenario in ETICS 1 : the user specifies in his/her profile that the required coverage tool during the build is EMMA Use case : the user specifies in his/her profile that the required coverage tool during the build is EMMA Test and Metrics Plug-in Framework’s runtime engine discovers dynamically and loads the „EMMA plug-in„Test and Metrics Plug-in Framework’s runtime engine discovers dynamically and loads the „EMMA plug-in„ (For example scans a directory named „plugins” looking for EMMA) (For example scans a directory named „plugins” looking for EMMA) In „plugins” directory the engine discovers „emma.jar”In „plugins” directory the engine discovers „emma.jar” When the build process reaches the „coverage” target the ‘emma.jar’ will be run using the configuration described in the manifest (xml) file of the EMMA pluginWhen the build process reaches the „coverage” target the ‘emma.jar’ will be run using the configuration described in the manifest (xml) file of the EMMA plugin

19 User scenario in ETICS 2 Use case : ETICS user wants to use TestManager to run gLite testcases outside the build process but using his/her own charting library to create reports In his profile (or somehow through the user interface) or from the user interface of the Plugin Framework he can chose TestManager between the available test driversIn his profile (or somehow through the user interface) or from the user interface of the Plugin Framework he can chose TestManager between the available test drivers The plugin manager observes that TestManager should be loadedThe plugin manager observes that TestManager should be loaded Also observes that when executing the reporting module in TestManager an extension of it e.g the charting system of the user should be loadedAlso observes that when executing the reporting module in TestManager an extension of it e.g the charting system of the user should be loaded

20 User scenario in ETICS 3 Use case: ETICS user (manager) wants to know how many open bugs were fixed during a specific build In his profile (or somehow through the user interface) he specifies the required metricIn his profile (or somehow through the user interface) he specifies the required metric The Plug-in Frameworks runtime engine discovers the required metric calculation utilityThe Plug-in Frameworks runtime engine discovers the required metric calculation utility When the build process reaches the target „metrics” the selected metric calculation utility will be run.When the build process reaches the target „metrics” the selected metric calculation utility will be run.

21 Test and Metrics Plug-in Architecture 1 Plugin Manager - loads and activates plugins - maintains a plug-in registry - finds the physical location of plugins EMMA JavaCov/ Cobertura API for defining Manifest files Manifest2.xml Manifestn.xml Manifest1.xml TestManager JPF

22 Test and Metrics Plug-in Architecture 2 Plugin Manager - loads and activates plugins - maintains a plug-in registry - finds the physical location of plugins JavaCov/ Cobertura API for defining Manifest files Manifest2.xml Manifestn.xml Manifest1.xml TestManager extension point Java Plugin Framework Build system - (build.xml) Charting system EMMA

23 Conclusion 2006 Specification on conceptual levelSpecification on conceptual level Feasibility studyFeasibility study Architectural specificationArchitectural specification Integration of the plugin system in the ETICS frameworkIntegration of the plugin system in the ETICS framework2007 ImplementationImplementation