1. EMI INFSO-RI-261611 ARC tools for revision and nightly functional tests Jozef Cernak, Marek Kocan, Eva Cernakova (P. J. Safarik University in Kosice, Kosice, Slovak Republic)

2. EMI INFSO-RI-261611 Testing strategy in ARC ARC-EMI testing tools – Build testing Nightly (every night) Revision (after each commit of developers) – Automatic functional testing of nightly builds Future plans – Increasing number of automatic functional tests – Automatic performance testing Conclusion Use Insert Header & Footer to set this field 2 Outline

3. EMI INFSO-RI-261611 Several roles in project: developers, testers and ARC release manager Adopted tests: – Unit tests(developers) – Build tests (testers) Revision (Kosice) Nightly (Copenhagen) EMI-RC (ETICS) – Functional tests (testers, developers) Manual Automatic (Kosice) – Regression testing (testers, developers) – Performance testing (testers) Manual Automatic in development phase – Large scale testing (in preparation phase) Use Insert Header & Footer to set this field 3 Testing practice in ARC

4. EMI INFSO-RI-261611 ARC-EMI testing tools Relation between ARC and EMI testing tools Use Insert Header & Footer to set this field 4

5. EMI INFSO-RI-261611 Block diagram of automatic revision and Functional tools Simple database server with PHP and Python scripts Use Insert Header & Footer to set this field 5

6. EMI INFSO-RI-261611 Infrastructure (local cluster) – 2 SUN 2200M2 servers (2x4GB RAM, 250GB disk, AMD 64 bits) – 2 desktops (2x 1GB RAM, 500GB disk 32 bits) – 1 MAC mini (2GB RAM, 160GB, 32 bits) Software – MySQL, PHP, Python – Code is free Distributed data processing – SVN Technical details Use Insert Header & Footer to set this field 6

7. EMI INFSO-RI-261611 Features – Supported are several platforms of OS: CentOS (32,64), Debian(32,64), and MAX OS (64) – Detailed identification of errors during building process Autogen Configuration make make dist Build Revision automatic testing Use Insert Header & Footer to set this field 7

8. EMI INFSO-RI-261611 – User friendly interface to define search criteria Daily Weakly Monthly Custom – Revision number, etc (see next examples) Direct connection with SVN Warning messages for developers if code is broken – CCCC metric – Quick mechanisms to identify breaking event …continued. Use Insert Header & Footer to set this field 8

9. EMI INFSO-RI-261611 Example of www interface for automatic testing tools Home page: http://arc-emi.grid.upjs.sk/ (Revision database is slow)http://arc-emi.grid.upjs.sk/ Use Insert Header & Footer to set this field 9

10. EMI INFSO-RI-261611 Several criteria to search results of revision testing Use Insert Header & Footer to set this field 10

11. EMI INFSO-RI-261611 Example of the results of daily revisions Use Insert Header & Footer to set this field 11

12. EMI INFSO-RI-261611 Features – Automatic download code from SVN based on several criteria: trunk, revision code, nightly builds – Automatic build – Automatic deployment – User friendly interface for: submitting proposals of test cases to search the results of functional tests – Tests are grouped into two main groups: functional tests of server, functional tests of client. Automatic functional tests Use Insert Header & Footer to set this field 12

13. EMI INFSO-RI-261611 Interface to enter search criteria Use Insert Header & Footer to set this field 13

14. EMI INFSO-RI-261611 The results of automatic functional tests Use Insert Header & Footer to set this field 14

15. EMI INFSO-RI-261611 client_arcsub_to_arex_gridmap_simplejob_jsdl – Tries to submit a simple JSDL job to a grid-map secured A-REX service. client_arcsub_to_grid-manager_gridmap_simplejob_jsdl – Tries to submit a simple JSDL job to a grid-map secured ARC0 CE. client_arcsub_to_GIIS_simplejob_jsdl – Submission of a simple JSDL job to a GIIS service. client_arcsub_to_ISIS_simplejob_jsdl – Submission of a simple JSDL job to an ISIS cloud. client_arcsub_random_broker_isis_simplejob_jsdl – Submission of a simple JSDL job to an ISIS cloud using random broker. client_arcsub_random_broker_giis_simplejob_jsdl – Submission of a simple JSDL job to an GIIS cloud using random broker. client_arcsub_fastestqueue_broker_isis_simplejob_jsdl – Submission of a simple JSDL job to an ISIS cloud using fastest queue broker. client_arcsub_fastestqueue_broker_giis_simplejob_jsdl – Submission of a simple JSDL job to a GIIS service using fastest queue broker. Failed – There was some error before we managed to start testing. The error could have been in code retrieval, installation or somewhere else. Functional test scenaria Use Insert Header & Footer to set this field 15

16. EMI INFSO-RI-261611 Requirements: – several servers and clients – Monitoring of several distributed processes: resources: – Memory – CPU – Network usage Performance – Scalability – Number of concurrent requests Reliability of services: – Ratio between successful tasks and total tasks. Examples of tasks: job submission, file transfer and etc. Automatic performance tests Use Insert Header & Footer to set this field 16

17. EMI INFSO-RI-261611 Proposed infrastructure: – Grid servers and clients, we plan to use existing infrastructure – Communication layer XML-RPC client-server architecture – Central database server – Common Python library on each server and client (for example start stop service) – Central control program … continued. Use Insert Header & Footer to set this field 17

18. EMI INFSO-RI-261611 Automatic testing covers: – Build process (ARC, ETICS), – Functional, – Performance tests (in progress). Utilization of automatic test tools increases probability to find weak part of the software. On the other hand, manual tests are important. Use Insert Header & Footer to set this field 18 Conclusions

19. EMI INFSO-RI-261611 Testing in ARC – http://wiki.nordugrid.org/index.php/Testing http://wiki.nordugrid.org/index.php/Testing Revision tests – http://download.nordugrid.org/builds/ http://download.nordugrid.org/builds/ – http://arc-emi.grid.upjs.sk/revisionTests.php http://arc-emi.grid.upjs.sk/revisionTests.php Functional tests – http://arc-emi.grid.upjs.sk/functionalTests.php http://arc-emi.grid.upjs.sk/functionalTests.php Code: – http://svn.nordugrid.org/trac/workarea/browser/AR CTestScriptshttp://svn.nordugrid.org/trac/workarea/ browser/ARCTestScripts http://svn.nordugrid.org/trac/workarea/browser/AR CTestScriptshttp://svn.nordugrid.org/trac/workarea/ browser/ARCTestScripts References Use Insert Header & Footer to set this field 19

20. EMI INFSO-RI-261611 Thank you Use Insert Header & Footer to set this field 20 Use Insert Header & Footer to set this field EMI is partially funded by the European Commission under Grant Agreement INFSO-RI-261611