SC7 WG6 Rome Engineering Ingegneria Informatica S.p.A. INFSO-RI Isabel Matranga ETICS Automated Building,Testing and Quality Assurance

ETICS INFSO-RI Rome, 19 October 2009 ETICS projects E-Infrastructure for Testing Integration and Configuration of Software

ETICS INFSO-RI Rome, 19 October 2009 ETICS automates and improves … ETICS automates and improves the execution of: Builds Tests Quality Verification … offered as a service to its users

ETICS INFSO-RI Rome, 19 October 2009 The context 4

ETICS INFSO-RI ETICS Keywords All-in-one solution building/testing/quality – Support for execution of continuous software build – Automatic creation of distributed packages – Dependency management Quality made easy – Verification of the quality of the software produced – Offers the Automated-Quality Certification Model compliant with ISO standards – Collection of test information Language / platform / tool independent – ETICS is non-intrusive and easily extensible – Plugin based system Distributed environment – Support for syncronization of developpers – Distributed builds and test – Multiplatform build and tests

ETICS INFSO-RI Rome, 19 October 2009 ETICS supports software professionals Requirements and Specification Requirements and Specification Maintenance Review and Improvement Maintenance Review and Improvement Architecture and Design Testing, Validation and Verification Deployment Implementation and Build Implementation and Build

ETICS INFSO-RI Rome, 19 October 2009 ETICS users Main users (apart from ETICS 2 project itself) “We use ETICS to build these software components across a variety of platforms, By using ETICS we are able to benefit from an automated build environment which can be reproducible at a later date” Steven Newhouse, EGEE III Technical Director “ETICS makes build and integration nightmare activity a simple configuration activity” Pasquale Pagano, D4Science Technical Director EDGeS “A-QCM certification, will contribute to the successful promotion of EDGeS achievements and solutions towards industrial partners” Prof. Peter Kacsuk, coordinator of EDGeS consortium

ETICS INFSO-RI Rome, 19 October 2009 What is A-QCM? How does ETICS work?

ETICS INFSO-RI ETICS System demo Automated Testing and Quality Assurance

ETICS INFSO-RI Table of Content Building with ETICS Testing with ETICS QA with ETICS

ETICS INFSO-RI Table of Content Building with ETICS Testing with ETICS QA with ETICS

ETICS INFSO-RI Architecture Behind the scene

ETICS INFSO-RI ETICS Portal Welcome

ETICS INFSO-RI WA Administration Security Authentication Access to ETICS services is based on X509 certificates and fine- grained user roles: Authorization based on RBAC model – Developers: can do local builds of project, subsystems and components – Integrators: can create and run integration configurations – Testers: can register and submit tests attached to configurations – Release Managers: can create and publish configurations, create release notes and other documentation – Project Managers: can create and manage projects, assign permissions, set policies, etc

ETICS INFSO-RI ETICS Portal The configurations

ETICS INFSO-RI Web Application Dependencies

ETICS INFSO-RI Web Application Remote build

ETICS INFSO-RI Repository

ETICS INFSO-RI Reports Presenting results Testing with ETICS

ETICS INFSO-RI Table of Content Building with ETICS Testing with ETICS QA with ETICS

ETICS INFSO-RI Build & Test packages Reports & Metrics Execution Engines AWS, Metronome, gLite, Unicore Repository WS - source code - test code continuous build and testing - job submission - deployment The ETICS vision in testing

ETICS INFSO-RI ETICS Testing services The multi-node testing functionality Automating execution of complex tests configuration: involving the deployment of several services, applications, clients on different hosts and their interaction ETICS supports natively several types of testing: Unit test - ETICS build and test system is designed to run the already existing unit tests, to collect the results in form of metrics but not to create new unit test cases Static test - this technique provides a foundation for producing solid code by exposing structural errors and preventing entire classes of errors. ETICS integrating static analysis tools and techniques into the build/test process can yield significant reductions in system testing and field failures

ETICS INFSO-RI Testing with ETICS The plugins The ETICS plugins are small units of functionality or wrappers around external tools Plugins are highly configurable in ETICS ETICS user can activate them according to policies called profiles. Profiles can be set as a static attribute of a configuration or dynamically using build properties The plugins collect information and publish data as Metrics Metrics are published with each build/test reports and are also stored individually in the ETICS Repository Optional plugins can be implemented and installed by users

ETICS INFSO-RI Plugins and Metrics Available plugins in ETICS: Sloccount (Single line of code count) – static analysis PyUnit, Junit, CppUnit – unit testing Pycoverage, Jcoverage Findbugs – static analysis CCCC IPv6, WSI CKJM Checkstyle PMD and more coming soon...

ETICS INFSO-RI Testing use case An ETICS success A typical scenario These are the typical issues of a complex system: Having a running DB Having a software dependency installed Running the test suites Accessing the reports A

ETICS INFSO-RI ETICS Answers Recently presented “Experiences with the ETICS support for testing in voms” at EGEE 09 faced the described scenario: ETICS can configure and running additional services during a build by adding a dependency on the third-party modules (like a DB) or towards a version of a software module ETICS provides two choices: ‘Test’ target in the “Build Commands Test Commands ETICS allows user-defined reports to be added to its own report page.

ETICS INFSO-RI Table of Content Building with ETICS Testing with ETICS QA with ETICS

ETICS INFSO-RI The ETICS vision in automating Quality Certification Defines metrics Runs measures Define thresholds Trials analysis ETICS grid infrastructure at CERN, INFN, UoW (NMI)

ETICS INFSO-RI A-QCM: Implementation QA EM METRICMETRIC Plugins

ETICS INFSO-RI A-QCM: Web interface report example Project level results Subsystem level results Component level results The result can be expanded to reach the metrics output related to the specific classes

ETICS INFSO-RI Future plans Really soon … Results from A-QCM trial certification on ETICS users’ software publicly available Enhancement of the multi-node testing feature More plug-ins for testing integrated in ETICS AWS cloud and Microsoft Azure Cloud under testing … In 2010 … ETICS offered as a service ETICS software available under Apache 2.0 license

ETICS INFSO-RI Thanks!