JRA1 Testing Current Status Leanne Guy Testing Coordination Meeting, 13 th September 2004 EGEE is a project funded by the European Union under contract IST

EGEE Testing Coordination Meeting - CERN, 7 September Outline Schedule and priorities CVS structure Tools and frameworks Test cases - examples Test reporting JRA1 testbed status Bugtracking

EGEE Testing Coordination Meeting - CERN, 7 September Schedule and priorities Service or componentExpected release date to testingRelative priority R-GMASeptember (PM06)High Logging and BookkeepingAugust (PM05)High Workload Management & Computing ElementSeptember (PM06)Very high Tactical Storage Element (SE)November (PM08)Medium Replica Catalog (RC)September 04 (PM06)Very high Metadata Catalog (MD)September 04 (PM06)Medium File Catalog (FC)October 04 (PM07)Medium Catalog services (CS)October 04 (PM07)Medium gLite IO (FIO)August 04 (PM05)Very high File transfer service (FT)September 04 (PM06)Medium File placement service (FP)September 04 (PM06)Medium VOMS ServerAugust 04 (PM05)High Services with securityOctober (PM07)Medium Grid Access Service (GAS)August 04 (PM05)High Accounting HLR server (HLR)October 04 (PM07)Medium Package manager (PM)October 04 (PM07)Medium

EGEE Testing Coordination Meeting - CERN, 7 September CVS structure CVS componentTests org.glite.testingTesting subsystem org.glite.testing.rgmaR-GMA tests org.glite.testing.dataData services tests org.glite.testing.securityVOMS tests org.glite.testing.wmsWorkload management system tests, org.glite.testing.lbLogging and bookkeeping tests org.glite.testing.gasGAS specific tests org.glite.testing.hlrHLR server tests org.glite.testing.package-managerPackage manager tests org.glite.testing.deploymentInstallation and configuration testing scripts org.glite.testing.interfaceInterface tests org.glite.testing.tools Reporting tools or any other custom utilities developed to assist the testing process, XML schemas for unit tests and test case and report definition org.glite.testing.reportsAutomatically generated test reports

EGEE Testing Coordination Meeting - CERN, 7 September CVS component Example: org.glite.testing.rgma  CVS  src :all test scripts for the rgma component  config:any configuration files, i.e. qmtest files, etc  doc:documentation for running the tests  LICENSE:The EGEE license  project:files needed by the build  build.xml:build.xml file  The src directory might contain subdirectories  Is there any interest in using a common CVS?  Is there any interest in building common “EGEE” test rpms?

EGEE Testing Coordination Meeting - CERN, 7 September Test Frameworks At the test case level, use xUnit framework to write tests Most tests to be written in Python making use of PyUnit to manage testcases and build testsuites xml output from Python using modifed unittest.py by Meb  Common xml output from all tests Higher level tool will be used to manage the execution of test suites: Still evaluating  QMtest :  NMI framework (Condor) Results will be written out into xml files Is there any interest in using a common framework?

EGEE Testing Coordination Meeting - CERN, 7 September R-GMA test example using PyUnit #!/usr/bin/env python import unittest, re,os, commands, string import edg.rgma as rgma class RGMAPyVersionTest(unittest.TestCase): def setUp(self): # Search pattern self.p = re.compile('^(#!/usr/bin/env)\s+(python\S*)') def tearDown(self): pass def testRgmaSetupScript(self): script = "/opt/glite/share/rgma/scripts/rgma-setup.py“ f = open(script,'r') l = f.readline() self.p.match(l) close(script) self.failUnlessEqual(self.p.match(l).group(2),"python") def testRgmaCLIScript(self): [snip] class RGMAPycTest(unittest.TestCase): def setUp(self): self.rpmlist = self.getRpmsInstalled("rgma") self.p = re.compile('^\S+(\.)(pyc)') def teardown(self): pass def testPycCodeExists(self): [snip] self.assertEqual(foundPycCode,"false") def suite(): suite1 = unittest.makeSuite(RGMAPyVersionTest) suite2 = unittest.makeSuite(RGMAPycTest) suite = unittest.TestSuite((suite1,suite2)) if __name__ == '__main__': """ Run all tests from command line """ # unittest.TextTestRunner(verbosity=2).run(suite()) unittest.main()

EGEE Testing Coordination Meeting - CERN, 7 September XML output from unittest Traceback (most recent call last):File "RGMAPythonTest.py", line 75, in testEdgRgmaScript raise AssertionError, "testEdgRgmaScript Failed"AssertionError: testEdgRgmaScript Failed

EGEE Testing Coordination Meeting - CERN, 7 September Test Cases Most test scripts will be written in Python,  Will not enforce a language to write tests in Will make use if WSDL definitions where possible Testing driven by the functional requirements from the NA4/PTF Testcase template defined but currently no test case specifications  Similar to NA4’s  Can we merge to a common format? Starting to look at defining test cases now that the design document is available and the service components/rpms are becoming available There is potentially a lot of overlap with NA4  Can we work together on test case definitions?

EGEE Testing Coordination Meeting - CERN, 7 September Test reporting and results Reporting tools  Tools to convert the XML output of all tests to HTML format  pdf resports can be produced as well. We will have a common reporting format for all tests. Installation testing reports page is a first example  testing/deployment_testing/installation/latest.html testing/deployment_testing/installation/latest.html Have other groups defined any reporting format ? Is is desireable to have a similar “look and feel”

EGEE Testing Coordination Meeting - CERN, 7 September JRA1 testing infrastructure 24 machines distributed across RAL, NIKHEF and CERN Running the gLite prototype since ~May Ad-hoc testing and development of an installation guide Migrating to EL now Component by component based installation  As new components are built on EL they are installed Not a single gLite installation yet Only have R-GMA, Castor and dCache SE installed and working on EL WMS installation going on now

EGEE Testing Coordination Meeting - CERN, 7 September Testing infrastructure

EGEE Testing Coordination Meeting - CERN, 7 September Bug tracking Savannah bug tracking being used. Categories defined for different components:  Prototype, gLite, Data, Alien, R-GMA, Security, WMS, LB, CE, Accounting.  Corresponds to high level gLite services All bugs associated with RH7.3 prototype go under ‘prototype’ All other bugs are submitted under the appropriate category Fixed bugs are moved to “Ready to test” and are automatically assigned to the ‘egeetest’ user Assign to one tester to assess

EGEE Testing Coordination Meeting - CERN, 7 September Bugtracking Testers should write test cases for the bug and commit to CVS If the bug is fixed and it was submitted by the tester, the tester will close the bug If it is fixed but was submitted by someone else, the bug will be assigned to the submitter to close If it is not fixed, its status is put back to “in progress” RPMs will be built that contain all the test cases and will be released.