1. SC7 WG6 Rome Engineering Ingegneria Informatica S.p.A. INFSO-RI-223782 Isabel Matranga ETICS Automated Building,Testing and Quality Assurance

2. ETICS INFSO-RI-223782 2 Rome, 19 October 2009 ETICS projects E-Infrastructure for Testing Integration and Configuration of Software

3. ETICS INFSO-RI-223782 3 Rome, 19 October 2009 ETICS automates and improves … ETICS automates and improves the execution of: Builds Tests Quality Verification … offered as a service to its users

4. ETICS INFSO-RI-223782 4 Rome, 19 October 2009 The context 4

5. ETICS INFSO-RI-223782 5 ETICS Keywords All-in-one solution building/testing/quality – Support for execution of continuous software build – Automatic creation of distributed packages – Dependency management Quality made easy – Verification of the quality of the software produced – Offers the Automated-Quality Certification Model compliant with ISO standards – Collection of test information Language / platform / tool independent – ETICS is non-intrusive and easily extensible – Plugin based system Distributed environment – Support for syncronization of developpers – Distributed builds and test – Multiplatform build and tests

6. ETICS INFSO-RI-223782 6 Rome, 19 October 2009 ETICS supports software professionals Requirements and Specification Requirements and Specification Maintenance Review and Improvement Maintenance Review and Improvement Architecture and Design Testing, Validation and Verification Deployment Implementation and Build Implementation and Build

7. ETICS INFSO-RI-223782 7 Rome, 19 October 2009 ETICS users Main users (apart from ETICS 2 project itself) “We use ETICS to build these software components across a variety of platforms, By using ETICS we are able to benefit from an automated build environment which can be reproducible at a later date” Steven Newhouse, EGEE III Technical Director “ETICS makes build and integration nightmare activity a simple configuration activity” Pasquale Pagano, D4Science Technical Director EDGeS “A-QCM certification, will contribute to the successful promotion of EDGeS achievements and solutions towards industrial partners” Prof. Peter Kacsuk, coordinator of EDGeS consortium

8. ETICS INFSO-RI-223782 8 Rome, 19 October 2009 What is A-QCM? How does ETICS work?

9. ETICS INFSO-RI-223782 9 ETICS System demo Automated Testing and Quality Assurance

10. ETICS INFSO-RI-223782 10 Table of Content Building with ETICS Testing with ETICS QA with ETICS

11. ETICS INFSO-RI-223782 11 Table of Content Building with ETICS Testing with ETICS QA with ETICS

12. ETICS INFSO-RI-223782 12 Architecture Behind the scene

13. ETICS INFSO-RI-223782 13 ETICS Portal Welcome

14. ETICS INFSO-RI-223782 14 WA Administration Security Authentication Access to ETICS services is based on X509 certificates and fine- grained user roles: Authorization based on RBAC model – Developers: can do local builds of project, subsystems and components – Integrators: can create and run integration configurations – Testers: can register and submit tests attached to configurations – Release Managers: can create and publish configurations, create release notes and other documentation – Project Managers: can create and manage projects, assign permissions, set policies, etc

15. ETICS INFSO-RI-223782 15 ETICS Portal The configurations

16. ETICS INFSO-RI-223782 16 Web Application Dependencies

17. ETICS INFSO-RI-223782 17 Web Application Remote build

18. ETICS INFSO-RI-223782 18 Repository

19. ETICS INFSO-RI-223782 19 Reports Presenting results Testing with ETICS

20. ETICS INFSO-RI-223782 20 Table of Content Building with ETICS Testing with ETICS QA with ETICS

21. ETICS INFSO-RI-223782 21 Build & Test packages Reports & Metrics Execution Engines AWS, Metronome, gLite, Unicore Repository WS - source code - test code continuous build and testing - job submission - deployment The ETICS vision in testing

22. ETICS INFSO-RI-223782 22 ETICS Testing services The multi-node testing functionality Automating execution of complex tests configuration: involving the deployment of several services, applications, clients on different hosts and their interaction ETICS supports natively several types of testing: Unit test - ETICS build and test system is designed to run the already existing unit tests, to collect the results in form of metrics but not to create new unit test cases Static test - this technique provides a foundation for producing solid code by exposing structural errors and preventing entire classes of errors. ETICS integrating static analysis tools and techniques into the build/test process can yield significant reductions in system testing and field failures

23. ETICS INFSO-RI-223782 23 Testing with ETICS The plugins The ETICS plugins are small units of functionality or wrappers around external tools Plugins are highly configurable in ETICS ETICS user can activate them according to policies called profiles. Profiles can be set as a static attribute of a configuration or dynamically using build properties The plugins collect information and publish data as Metrics Metrics are published with each build/test reports and are also stored individually in the ETICS Repository Optional plugins can be implemented and installed by users

24. ETICS INFSO-RI-223782 24 Plugins and Metrics Available plugins in ETICS: Sloccount (Single line of code count) – static analysis PyUnit, Junit, CppUnit – unit testing Pycoverage, Jcoverage Findbugs – static analysis CCCC IPv6, WSI CKJM Checkstyle PMD and more coming soon...

25. ETICS INFSO-RI-223782 25 Testing use case An ETICS success A typical scenario These are the typical issues of a complex system: Having a running DB Having a software dependency installed Running the test suites Accessing the reports A

26. ETICS INFSO-RI-223782 26 ETICS Answers Recently presented “Experiences with the ETICS support for testing in voms” at EGEE 09 faced the described scenario: ETICS can configure and running additional services during a build by adding a dependency on the third-party modules (like a DB) or towards a version of a software module ETICS provides two choices: ‘Test’ target in the “Build Commands Test Commands ETICS allows user-defined reports to be added to its own report page.

27. ETICS INFSO-RI-223782 27 Table of Content Building with ETICS Testing with ETICS QA with ETICS

28. ETICS INFSO-RI-223782 28 The ETICS vision in automating Quality Certification Defines metrics Runs measures Define thresholds Trials analysis ETICS grid infrastructure at CERN, INFN, UoW (NMI)

29. ETICS INFSO-RI-223782 29 A-QCM: Implementation QA EM METRICMETRIC Plugins

30. ETICS INFSO-RI-223782 30 A-QCM: Web interface report example Project level results Subsystem level results Component level results The result can be expanded to reach the metrics output related to the specific classes

31. ETICS INFSO-RI-223782 31 Future plans Really soon … Results from A-QCM trial certification on ETICS users’ software publicly available Enhancement of the multi-node testing feature More plug-ins for testing integrated in ETICS AWS cloud and Microsoft Azure Cloud under testing … In 2010 … ETICS offered as a service ETICS software available under Apache 2.0 license

32. ETICS INFSO-RI-223782 32 Thanks! http://www.eticsproject.eu