1. INFSO-RI-223782 Quality Assurance with ETICS – multi- node automated testing CGW 09 M.Zurek, A. A. Rodriguez, A. Aimar, A. di Meglio, L. Dini CERN Krakow, 12 October 2009

2. INFSO-RI-223782 2 Contents ETICS Overview Multi-Node Testing Q & A ETICS Testing ToolsCGW09

3. INFSO-RI-223782 3 ETICS Overview ETICS Testing ToolsCGW09

4. INFSO-RI-223782 The Problem 4 ETICS Online tutorials - Module 01 Distributed Computing, Distributed Development

5. INFSO-RI-223782 The ETICS System 5 ETICS Testing ToolsCGW09 Continuous software build, test and QA verification system Easily extensible with additional plugins Scheduled or on-demand build and test jobs User computers or remote distributed infrastructures Built-in connectors to distribute build and test jobs on different types of infrastructures from standard job management systems to the grid Open repository of configuration metadata, packages and build, test and QA reports Support for standard package management systems like YUM It’s multi-platform and independent from any specific language, build or test tool

6. INFSO-RI-223782 Architecture 6 CGW09ETICS Testing Tools

7. INFSO-RI-223782 The Configuration Web Client 7 ETICS Online tutorials - Module 01

8. INFSO-RI-223782 Command-line Client 8 ETICS Online tutorials - Module 01

9. INFSO-RI-223782 The Repository Web Application 9 ETICS Online tutorials - Module 01

10. INFSO-RI-223782 Examples of Plugins 10 ETICS Online tutorials - Module 01 MetricsTypeProgramming languages/ technologies ToolETICS Plugin ComplexitystaticJava Python JavancssJCcnPlugin PyComplexityPlugin.py Design qualitystaticJavaJdependJDependPlugin N of potential bugsstaticC/C++ Python Perl PHP Java Flawfinder, RATS PMD Findbugs CFlawfinderPlugin CPyPhpRatsPlugin JPmdPlugin JFindbugsPlugin N of potential bugsdynamicC/C++ValgrindCValgrindPlugin Lines of codestaticAllSLOCCountSLOCCountPlugin CoveragedynamicJavaEmma Cobertura JUnitemmaPlugin JCoberturaPlugin Unit tests success rate dynamicJava Python JUnit PyUnit JUnitPlugin JUnitreportsPlugin.py PyUnitPlugin.py Compliance with standards staticIPv6 WSI IPv6Plugin WSInteroperabilityPlugi n ProfilingdynamicC/C++ Java Jrat Valgrind JRatPlugin CValgrindPlugin

11. INFSO-RI-223782 11 Multi Node ETICS Testing ToolsCGW09

12. INFSO-RI-223782 12 Multi-Node ETICS Testing Tools The ability to setup scenarios where multiple services are automatically deployed in multiple nodes. These services must be able to work properly as if they would be installed manually by a user performing any required operation in the required order. CGW09

13. INFSO-RI-223782 13 Challange Service Synchronization ETICS Testing ToolsCGW09

14. INFSO-RI-223782 14 Challange Information Interchange ETICS Testing ToolsCGW09

15. INFSO-RI-223782 15 Challange Flow Monitoring ETICS Testing ToolsCGW09

16. INFSO-RI-223782 16 The ETICS Solution ETICS-SET is used to publish key-value pairs or only keys to the central information system in order to make the information available to all the other nodes, services or phases. ETICS-GET is used to get (blocking or not-blocking) the information published by some local or remote execution of ETICS-SET. ETICS-LOG is used to log and for real-time monitoring. ETICS Testing Tools Server NodeClient NodeCentral IS [ … INSTALL …] [ … CONFIG … ] [ … START … ] ETICS-SET S.IP = x.x.x.x ETICS-LOG “RUNNING” ETICS-GET C.DONE [… STOP AND EXIT …] $IP = ETICS-GET S.IP [ … USE $IP … ] [… RUN TEST …] ETICS-SET C.DONE [… EXIT …] S.IP = x.x.x.x S: “RUNNING” C.DONE CGW09

17. INFSO-RI-223782 The PING Example (1/3) ETICS Testing ToolsCGW09 17

18. INFSO-RI-223782 The PING Example (2/3) ETICS Testing ToolsCGW09 18

19. INFSO-RI-223782 The PING Example (3/3) CLIENT ETICS Testing ToolsCGW09 SERVER 19

20. INFSO-RI-223782 Challenges Deployment automation in the context of complex software development is a prerequisite for automatic system testing Complexity of service configuration Developers should provide deployable configurations, or parameterised configurations External dependencies: all dependencies must be deployable automatically Knowledge of these “external” components is not necessarily with the end-user Privileged deployment/configuration operations Root / Super User actions Mix of normal user-level and root level actions 20 ETICS Testing ToolsCGW09

21. INFSO-RI-223782 Benefits Automation of complex processes Automatic setup of complex environment Automatic generation of reports Save on resources and time More scenarios can be explored Less human errors Form the foundation for more advanced system tests 21 ETICS Testing ToolsCGW09

22. INFSO-RI-223782 Thanks! 22 http://www.eticsproject.eu ETICS Testing ToolsCGW09