Geant4: System Testing Team Steve O’Neale CERN IT/ASD and University of Birmingham LAL-Orsay, France 16-21 Oct. 2000 1/18/2019

STT Activities Shrunk from 14 names to 6 yielding 1 fte Bonsai recently introduced (Serguei Sadilov) gui to track state of testing from developers cvs tag internal proposed for testing selected for testing accepted rejected global Testing [the subject of this presentation] Tinderbox, to view results, next (Fred Jones) LXR to browse source code in “internal” use 1/18/2019

Current Platforms At CERN from stesting account can login from fukui, babar-online, G4Orsay etc. DXplus(5) afs iso/non-iso opt/deb 4 HPplus (5) afs opt/deb 2 SUNgeant nfs opt/deb 2 PCgeant nfs Linux opt/deb 2 NT (LHCb on reftags) 1 1/18/2019

Platforms: January 2001 CERN RISC Reduction Planning DX plus Frozen, only for LEP use HP plus Frozen, only for LEP use NT (PCSF) convert to Linux (after Feb LHCb production, if linux drivers available) LXBatch many machines SUN adequate for development Atlas, Alice and CMS to prepare requirements GEANT4 CB/TSB Change Supported Platforms ? Drop some risc architectures or ask principal users to host tests Add (maybe) newer SUN system, newer Linux, IA-64. 1/18/2019

STT Procedures (before bonsai) Documented with few changes Worked at the HEAD (with bonsai we work from a list of tags) prod and two dev areas, per platform, in afs at cern Some links to local disks (need more ) cvs update for tags prepared by hand make and scripts compile, make libraries, executables, run tests and diff outputs data file prepared by hand to drive scripts for each/all platforms may submit all libraries and/or tests to build and run or select just one test at high statistics (Large_N) inspect results (1000 file in three different file systems) report by mail and editing web page (updated on cvs commit) 1/18/2019

Automation and Monitoring(1) Prototypes in perl (in cvs, undocumented), no major changes to previous scripts, now need to change procedures, review platforms, sites and stt activities, specify reporting to Tinderbox. In prod/dev1/dev2 dir, source setup.©sh to establish environment. Hasami.plx queries bonsai database to extract last reference tag, accepted tags and tags for testing generates serial number (123), driver files for testing (reftag+tags123) prunes bonsai123.sdb for current e&o in bonsai, hosts123.data according to available test platforms run existing update.sh script yielding one line per file update123.log ReviewUpdate.plx check update log (for clashes, modified code, errors) produce short summary, could launch tests if no problems found we sometimes debug or work interactively in the source tree ExtractLastLog.plx FilterMake.plx ShowFiles.plx 1/18/2019

Automation and Monitoring (2) Launch tests g4allsys.sh < hosts123.data ExtractLastLog.plx (not a great name) inspects gmake.log for compilation and build (size and age) g4allsys.sh log for tests (Started, Finished, Disabled etc) generates web page could be run (every n minutes) via acrontab at cern. FilterMake.plx more sophisticated than grep (E)rror attempts to classify each line in gmake.log might be developed to tabulate known and new warning does Tinderbox already have such analysis tools ? 1/18/2019

Automation and Monitoring (3) ShowFiles.plx examines the expected test log (and diff) files reports missing files (i.e. test not run) reports G4 error messages per test excludes differences wrt reference output due to cvs version tags, directories used, date, time, iso/non-iso formatting, some cases where many tracking steps are printed and so on shows number of differences (if any) per test per platform 1/18/2019

STT results for the developer Tag is accepted or rejected may reverts to proposed if co-working tags need work. Compilation or load failure on developer’s least favourite platform small error can generate 20Mb of logs for hp “future” compiler Tests results run to completion - no change wrt reference output run to completion - developer validates change to reference output (new) warnings and/or fatal errors found test loops or exhausts resources in some way 1/18/2019

STT Operational Problems Few tests of a tagset have finished cleanly in 2000 Modifying the scripts to make use of local (nfs mounted) disk space on PUBLIC machines will help a lot. Independent monitoring is needed as there are many ways in which NO output is obtained machine re-booted to fix (say) an afs hang in atlas/delphi build machine down, has too many processes, disks full, token expires test program loops, or waits, or fills disks (selective switched off) afs can hang waiting for a server response at any time from after 10 seconds when clearing old test files (rm bin/*) to writing (ld) the last test executable after hours of building 1/18/2019

Next Steps Bonsai, tinderbox, lxr, changing the testtools, monitoring and automating the procedures, as suggested, can reduce the stt manpower needs and improve the presentation to the developers. Before proceeding, we have some questions for G4 Will we soon drop platforms (Dec, Hp, NT) ? Will we have test sites other than CERN ? Will we have more than 1 person to run the tests ? We could then take on more interesting activities timing, profiling, leak detection, code metrics, regression analysis, extend test cases to more physics and detectors ….. 1/18/2019