Software Status/Plans Torre Wenaus, BNL/CERN US ATLAS Software Manager US ATLAS PCAP Review November 14, 2002

Torre Wenaus, BNL/CERN PCAP Review, November 14, 2002 Slide 2 U.S. ATLAS Software Project Overview  Control framework and architecture  Chief Architect, principal development role, ATLAS LCG applications area liaison  Databases and data management  Database Leader, primary ATLAS expertise on ROOT/relational baseline  Software support for development and analysis  Software librarian, quality control, software development tools, training…  Automated build/testing system adopted by Int’l ATLAS  Subsystem software roles complementing hardware responsibilities  Muon system software coordinator  Scope commensurate with U.S. in ATLAS: ~20% of overall effort  Commensurate representation on steering group  Strong role and participation in LCG common effort Recent developments in green

Torre Wenaus, BNL/CERN PCAP Review, November 14, 2002 Slide 3 U.S. ATLAS Software Organization

Torre Wenaus, BNL/CERN PCAP Review, November 14, 2002 Slide 4 U.S. ATLAS - ATLAS Coordination US roles in Int’l ATLAS software: D. Quarrie (LBNL), Chief Architect D. Malon (ANL), Database Coordinator P. Nevski (BNL), Geant3 Simulation Coordinator, Simulation production lead C. Tull (LBNL), EDG WP8 Liaison H. Ma (BNL), Raw Data Coordinator T. Wenaus (BNL), Planning Officer USInternational See task matrix

Torre Wenaus, BNL/CERN PCAP Review, November 14, 2002 Slide 5 ATLAS Subsystem/Task Matrix Offline Coordinator ReconstructionSimulationDatabase ChairN. McCubbinD. RousseauA. Dell’Acqua D. Malon Inner DetectorD. BarberisD. RousseauF. Luehring S. Bentvelsen / D. Calvet Liquid ArgonJ. CollotS. RajagopalanM. Leltchouk H. Ma Tile CalorimeterA. SolodkovF. MerrittV.TsulayaT. LeCompte MuonJ.ShankJ.F. LaporteA. RimoldiS. Goldfarb LVL 2 Trigger/ Trigger DAQ S. GeorgeS. TapproggeM. Weilers A. Amorim / F. Touchard Event FilterV. VercesiF. Touchard Computing Steering Group members/attendees: 4 of 19 from US (Malon, Quarrie, Shank, Wenaus) Physics Coordinator: F.Gianotti Chief Architect: D.Quarrie

Torre Wenaus, BNL/CERN PCAP Review, November 14, 2002 Slide 6 Project Planning Status  U.S./Int’l ATLAS WBS/PBS and schedule fully unified  US/Int’l software planning covered by same person (TW)  Synergies outweigh the added burden of the ATLAS Planning Officer role  No ‘coordination layer’ between US and Int’l ATLAS planning  Possible because of how the ATLAS Planning Officer role is currently scoped  As pointed out by an informal ATLAS computing review in March, ATLAS would benefit from a full FTE Planning Officer  I have a standing offer to the Computing Coordinator: to step aside if/when a capable person with more time is found  Until then, I scope the job to what I have time for and what is highest priority

Torre Wenaus, BNL/CERN PCAP Review, November 14, 2002 Slide 7 ATLAS Computing Planning  US led a comprehensive review and update of ATLAS computing schedule in the spring  Milestone count increased by 50% to 600; many others updated  Milestones and planning coordinated around DC schedule  Reasonably comprehensive and detailed through 2002  New round underway now to flesh out 2003 schedule  US core activity scheduling in reasonable shape  Long term milestones reworked to reflect LHC schedule, LCG  Centered around escalating data challenges  Weak decision making, still a problem, translates to (among other things) weak planning  Strong recommendation of the March review to fix this  Should be fixed in the present computing management reorganization

Torre Wenaus, BNL/CERN PCAP Review, November 14, 2002 Slide 8 Summary Major Milestones Green: Done Gray: Original date Blue: Current date

Torre Wenaus, BNL/CERN PCAP Review, November 14, 2002 Slide 9 Major Milestones One DC per year until startup

Torre Wenaus, BNL/CERN PCAP Review, November 14, 2002 Slide 10 Data Challenge 1  DC1 phase 1 (simu production for HLT TDR) executed successfully  World-wide operation  Phase 2, starting in the next weeks, focuses on testing of new software  Introduction and testing of new event data model  Intensive Geant4 usage  Intensive Grid usage  Data production for Physics and computing model studies  First tests of computing model  Analysis using Analysis Object Data (AOD)  DC1 Phase 2 too early for LCG common persistency POOL production use  POOL production releases in spring/summer 2003

Torre Wenaus, BNL/CERN PCAP Review, November 14, 2002 Slide 11 Software Support, Quality Control  New releases are available in the US typically ~1-2 days after CERN  Provided in AFS for use throughout the US  Librarian receives help requests and queries from ~25 people in the US  US-developed nightly build facility used throughout ATLAS  Central tool in the day to day work of developers and the release process  Recently expanded as framework for progressively integrating more quality control and testing  Testing at component, package and application level  Code checking to be integrated  CERN support functions being transferred to new ATLAS librarian  BNL-based nightlies recently resumed  Much more stable build environment than CERN at the moment  Use timely, robust nightlies to promote usage of the Tier 1 for development

Torre Wenaus, BNL/CERN PCAP Review, November 14, 2002 Slide 12 Software Support, Quality Control (2)  Testing integrated into automated builds  Unit tests, package tests, integration/system tests  ATLAS has (finally!) established a dedicated support team (SIT) for software infrastructure, testing, release management etc.  U.S. represented by the U.S. ATLAS librarian, an active team member  SIT needs a dedicated leader (currently the rotating, and overloaded, release manager heads SIT)  Provides a much needed context for U.S. support and QA efforts  pacman (Boston U) for remote software installation  Adopted by grid projects for VDT, and a central tool in US grid testbed work

Torre Wenaus, BNL/CERN PCAP Review, November 14, 2002 Slide 13 Grid Software  Software development within the ATLAS complements of the grid projects is being managed as an integral part of the software effort  Grid software activities tightly integrated into ongoing core software program, for maximal relevance and return  Grid project programs consistent with this have been developed  And has been successful  e.g. Distributed data manager tool (Magda) we developed was adopted ATLAS-wide for data management in the DCs  Grid goals, schedules integrated with ATLAS (particularly DC) program  However we do suffer some program distortion  e.g. we have to limit effort on providing ATLAS with event storage capability in order to do work on longer-range, higher-level distributed data management services

Torre Wenaus, BNL/CERN PCAP Review, November 14, 2002 Slide 14 Effort Level Changes  ANL/Chicago – loss of.5 FTE in DB  Ed Frank departure; no resources to replace  Another.5 FTE recently lost, will be replaced  BNL – cancelled 1 FTE new hire in data management  Insufficient funding in the project and the base program to sustain the bare- bones plan  Results in transfer of DB effort to grid (PPDG) effort – because the latter pays the bills, even if it distorts our program towards lesser priorities  LBNL – stable project-supported FTE count in architecture/framework  But loss of base support is threatening effort level and deliverables  Grid funding being sought to ameliorate  DB effort hard-hit, but ameliorated by common project  Because the work is now in the context of a broad common project, US can still sustain our major role in ATLAS DB  A material example of common effort translating into savings (even if we wouldn’t have chosen to structure the savings this way!)

Torre Wenaus, BNL/CERN PCAP Review, November 14, 2002 Slide 15 Personnel Priorities for FY02, FY03  This is how we are doing relative to goals…  Sustain LBNL (4.5FTE) and ANL (3FTE) support  This we are doing so far.  Add FY02, FY03 1FTE increments at BNL to reach 3FTEs  Failed in 02; BNL hire cancelled. Should recover to 3 FTEs in FY03  Restore the.5FTE lost at UC to ANL  No resources  Establish sustained presence at CERN.  No resources, despite being a very high priority  We rely on labs to continue base program and other lab support to sustain existing complement of developers  And needed base program support is not there. Lab base programs are being hammered.

Torre Wenaus, BNL/CERN PCAP Review, November 14, 2002 Slide 16 General and longer term priorities  These are reflected in the software request in the research program proposal (and go back as far as our original Jan 2000 project plan)  Priorities in order:  Sustain existing ANL, BNL, LBNL efforts  Complete the ramp of the lab based core developer FTEs to the long-planned levels  ANL 3.5 FTEs, LBNL 4.5 FTEs, BNL 4 FTEs  Establish, over and above these lab levels, presence at CERN of at least 2 core developer FTEs  In addition to any lab people who might be located at CERN  Establish effort at the core-subsystem interface – sited mainly at universities and possibly CERN – to  support the translation of core developments into established software employed by end users  better support the leadership roles held by the US with developer effort capable of translating decisions into established solutions

Torre Wenaus, BNL/CERN PCAP Review, November 14, 2002 Slide 17 SW Funding Profile Comparisons 2000 agency guideline January 2000 PMP 11/2001 guideline ‘Compromise profile’ requested in 2000 Mid 02 bare bones

Torre Wenaus, BNL/CERN PCAP Review, November 14, 2002 Slide 18 Concluding Remarks  No strategic changes; program is working, but stressed by funding  US has consolidated the leading roles in our targeted core software areas  Involved with new LCG common efforts in all our core areas  Architecture/framework effort level being sustained so far  And is delivering the baseline core software of ATLAS  Database/data mgmt effort reduced but so far preserving our key technical expertise  Leveraging that expertise for a strong role in common project  Cannot tolerate further reduction in a key strategic US core area  US major contributor to software infrastructure and QA in ATLAS  Recent emphasis: improve QA, and make the US development and production environment as effective as possible  Soft support from the project and base programs while the emphasis on grids grows is distorting our program in a troubling way