1. Nightly Releases and Testing Alexander Undrus Atlas SW week, May 24 2004

2. Atlas SW week, May 24, 04 – Alex Undrus – Nightly Builds2 Outline Status of Nightly Builds New NICOS version 0.3 Testing in Nightly Builds

3. Atlas SW week, May 24, 04 – Alex Undrus – Nightly Builds3 Status of Nightlies Three nightlies: opt, dbg, “8.0.x” 2x2GHz machines: 6 hrs gain: Make is faster by factor 1.5 – 2.0 Builds are finished by ~ noon (CERN time) More efforts for acceleration Parallel checkout: saves 1 hour but connection problems with CVS server Multithreaded builds: new tests with CMT v1r16 start soon Incremental builds (Simon George)

4. Atlas SW week, May 24, 04 – Alex Undrus – Nightly Builds4 Plan for better robustness of Nightlies Builds on local disk followed by copy to AFS Better stability, decreasing AFS load Planned setting: Release is checked out to local disk Opt & dbg builds start simultaneously Interlock mechanism for building NOVA Copy to AFS (script copy_release available) Waiting Database team for implementation of NOVA interlocks

5. Atlas SW week, May 24, 04 – Alex Undrus – Nightly Builds5 Purpose of NICOS NIghtly COntrol System Provide flexible nightly build system for coordination of distributed development Test new software submissions Send immediate feedback to developers Facilitate multi-person, multi-platform development of code Portable System (perl based)

6. NICOS Organization NICOS Controller NICOS Job Error Analysis Testing Project Make Project Setup Code Checkout Packages Database Packages Database Configur. Cache NICOS Webmaster Version Admin. Atlas SW week, Dec.03 - Alex Undrus - NICOS 7

7. Atlas SW week, May 24, 04 – Alex Undrus – Nightly Builds7 History of NICOS Fall 2001: start of ATLAS nightly builds with set of shell scripts Jan 2003: start of development of NICOS, perl based nightly control system Feb 2003: version 0_1 of NICOS is released and applied for POOL nightly builds (LCG project) July 2003: NICOS is accepted as LCG nightly build tool, used in POOL and SEAL builds Nov 2004: NICOS version 0_2 applied for ATLAS builds

8. Atlas SW week, May 24, 04 – Alex Undrus – Nightly Builds8 NICOS version 0.3 Improved pages design: Tag list emphasizes new packages and versions Three sorting options for list of packages Added category of QA tests (to previously used integrated and unit tests) Test results on separate web page, while page with build results contain test summary Added “warning” level for minor problems Builds “on demand” (use NICOS in users work environment), tested by S. George

9. Atlas SW week, May 24, 04 – Alex Undrus – Nightly Builds9 NICOS version 0.3 (cont.) Build results available immediately after make loop Integration with QMTest, Python based testing tool, supported by LCG

10. NICOS Project Web Page Atlas SW week, Dec.03 - Alex Undrus - NICOS 12 new

11. NICOS Tag List Atlas SW week, Dec. 2003 - Alex Undrus - NICOS 13 NEW

12. NICOS Build Results Atlas SW week, Dec.03 - Alex Undrus - NICOS 14 NEW

13. NICOS Test Results CHEP’03 - Alexander Undrus - NICOS system 15 NEW

14. Atlas SW week, May 24, 04 – Alex Undrus – Nightly Builds14 Nightly Testing Principles Automation Smoke tests Support different granularity levels QA tests Unit (structural) tests Integrated (behavioral) tests Use of test tools (CppUNIT, QMTest) Convenience for developers Fast feedback to users

15. Atlas SW week, May 24, 04 – Alex Undrus – Nightly Builds15 QMTEST Open-source, cross-platform testing tool Python based, extensions possible Tests are described in xml format Organizes tests hierarchically (in suites) Provides a graphical interface for creating and running tests Also can be used from command line

16. Atlas SW week, May 24, 04 – Alex Undrus – Nightly Builds16 QMTEST & NICOS integration Integration is provided by Python script (qmtest_config): Searches for test scripts NICOS test database Release tree (in directories test) Determines test suites NICOS test database Pattern “TEST_SUITE” in script body Tries to guess Creates XML descriptions for QMTest

17. Atlas SW week, May 24, 04 – Alex Undrus – Nightly Builds17 QMTEST & NICOS integration (cont.) QMTest runs tests, determines results (regression is possible), handles log info Results are reflected on NICOS pages and also can be browsed with QMTest graphical interface Test can be rerun from QMTest graphical interface

18. NICOS & QMTEST integration NICOS Job Test Database Test Database NICOS Webmaster Atlas SW week, Dec.03 - Alex Undrus - NICOS 7 qmtest_config.py QMTEST XML test descriptions

19. QMTEST page CHEP’03 - Alexander Undrus - NICOS system 15

20. QMTEST suite CHEP’03 - Alexander Undrus - NICOS system 15

21. Atlas SW week, May 24, 04 – Alex Undrus – Nightly Builds21 CppUnit Tool for unit testing of C++ code/classes allows to run tests automatically and gives a report about successes and failures For developers that write software and test simultaneously ATLAS TestPolicy provides access to LCG installation of CppUnit and integrates it in CMT environment (since January 2004). Details: http://atlas.web.cern.ch/Atlas/GROUPS/SOFTWARE/ OO/dist/nightlies/nicoswww/cppunit_testing.html

22. Atlas SW week, May 24, 04 – Alex Undrus – Nightly Builds22 Status of NICOS 0.3 Trial 8.1.0 builds on lxbuild005 local disk: Since ~ May 10 atlas.web.cern.ch/Atlas/GROUPS/SOFTWARE/OO/dist/nightli es/testww Only one significant problem revealed: QMTest does not work with PyXML used by ATLAS Temporary solution: exclusion of PyXML from PATHs Test in real nightlies is scheduled on the next weekend Update of manual to be completed by June 1

23. Atlas SW week, May 24, 04 – Alex Undrus – Nightly Builds23 To Do list for NICOS Improve convenience features for “builds on demand” Create scripts for administrative tasks for stable releases (rebuilds, tests, documentation)

24. Atlas SW week, May 24, 04 – Alex Undrus – Nightly Builds24 Further Information ATLAS nightly builds web page atlas.web.cern.ch/Atlas/GROUPS/SOFTWARE/OO/dist /nightlies/atlaswww/nightly_builds.html atlas.web.cern.ch/Atlas/GROUPS/SOFTWARE/OO/dist /nightlies/atlaswww/nightly_builds.html NICOS web page www.usatlas.bnl.gov/computing/software/nicos/index.html www.usatlas.bnl.gov/computing/software/nicos/index.html QMTest integration page http://atlas.web.cern.ch/Atlas/GROUPS/SOFTWARE/ OO/dist/nightlies/nicoswww/qmtest_atlas.html