October 23, 2000USCMS S&C Project Matthias Kasemann1 US CMS Software and Computing The Project Plan Matthias Kasemann Fermilab FNAL Oversight Panel October 23, 2000

USCMS S&C Project Matthias Kasemann2 Talk Outline Talk Outline  Overview of the US CMS Project Management Plan è Scope of the US CMS S&C project è Organization of the US CMS S&C Project è Relation to Int’l CMS è Status and Future Evolution of the PMP  Current status of the US CMS S&C Project

October 23, 2000USCMS S&C Project Matthias Kasemann3 LHC Software & Computing Challenges è Geographical dispersion: of people and resources è Complexity: the detector and the LHC environment è Scale: Petabytes per year of data 1800Physicists (360 from US) 150 Institutes 32 Countries Major challenges associated with:  Coordinated Use of Distributed computing resources  Remote software development and physics analysis  Communication and collaboration at a distance R&D: New Forms of Distributed Systems: Data-Grids

October 23, 2000USCMS S&C Project Matthias Kasemann4 US and Worldwide Data Grid Tier2 Center Online System Offline Farm, CERN Computer Center FNAL Center France Center Italy Center UK Center Institute Institute ~0.25TIPS Workstations ~100 MBytes/sec ~2.4 Gbits/sec Mbits/sec Bunch crossing per 25 nsecs. 100 triggers per second Event is ~1 MByte in size Physicists work on analysis “channels”. Each institute has ~10 physicists working on one or more channels Physics data cache ~PBytes/sec ~ Gbits/sec + Air Freight Tier2 Center ~622 Mbits/sec Tier 0 +1 Tier 1 Tier 3 Tier 4 Tier2 Center Tier 2 Experiment

October 23, 2000USCMS S&C Project Matthias Kasemann5 The Goal of the “S&C Project”  To provide the software and computing resources needed to enable US physicists to fully participate in the physics program of CMS  Allow US physicists to play key roles and exert an appropriate level of leadership in all stages of computing related activities … è From software infrastructure to reconstruction to extraction of physics results è From their home institutions, as well as at CERN This capability should extend to physicists working at their home institutions

October 23, 2000USCMS S&C Project Matthias Kasemann6 US-CMS Software and Computing Project  Aug 1998 First presentation to DOE/NSF (Newman + Taylor)  12/98 - 6/00Project Management Plan (PMP)  11/98, 5/99 Favorable DOE/NSF “Peer” Reviews; Funding for Software Engineers  11/ /99Biannual Reports and Discussion at JOG  7/99 - Acting Software and Computing Board; L1/L2 PMs  12/99 Submission of Draft PMP to DOE/NSF  1/00WBS, Draft Funding Profile for Presented at DOE/NSF Peer Review  3/00Start Up Funding for UF Subproject  5/17/00GriPhyN ITR Proposal (Grid IT R&D)  5/20/00SC Organization Approved and PMP by US CMS CB  5/30/00DOE/NSF Proposed Funding Profile  6/12/00US CMS Response, Discussion at JOG  6/25/00Finalize updated PMP, sent to collaboration  6/29/00 Quarterly Progress Report at DOE  6/ /00Update Requirements and Costs; Refine WBS  7/ /00Plan Tier2 Center Prototype Startup  11/14/00Project Baselining Review by DOE/NSF  11/00L1 PM Starts; Appoints L2 PMs (CAS, UF)

October 23, 2000USCMS S&C Project Matthias Kasemann7 Outline of Software and Computing Plan  Goals of the Project  Project Organization  Upper Level Project Management  Interrelationship with Other Entities  Evolution of the US CMS Software and Computing Project p Newest Version is October 20, 2000 (separated Project Plan Management Plan from WBS, milestones and budget)  Additional Documents describe: è High Level Milestones, Work Breakdown Structure, and Budget for the User Facilities Subproject è High Level Milestones, Work Breakdown Structure, and Budget for the Core Applications Subproject è High Level Milestones, Work Breakdown Structure, and Budget for Overall Project è Appendix: p CMS Software Milestones and WBS

October 23, 2000USCMS S&C Project Matthias Kasemann8 Participants/Key Players  CERN  International CMS  US CMS  US Funding agencies  US Universities and National Labs  US Physicists

October 23, 2000USCMS S&C Project Matthias Kasemann9 Key Management Entities è Level 1 Project Manager + deputy è Level 2 Project Manager for Core Applications Software è Level 2 Project Manager for User Facilities è The Advisory Software and Computing Board -- ASCB è The Fermilab Director or Designee advised by the Project Management Group è The Fermilab Computing Division è US funding agencies and Joint Oversight Group

October 23, 2000USCMS S&C Project Matthias Kasemann10 US Funding Agencies and Fermilab (CMS Host Institution) Fiscal Authority Policy & reporting Software and Computing Project Project Manager Liaison Core Applications Software User Facilities Support US CMS Advisory Software and Computing Board (USASCB) Reconstruction Detector Software and Physics Groups Organization of US CMS S&C Project

October 23, 2000USCMS S&C Project Matthias Kasemann11 US CMS Software and Computing Scope  The US CMS Software and Computing Project è Core Application Software è User Facilities p Tier 1 and Tier 2 regional Centers  Networking support è CERN--US/Fermilab è Internal to US  Reconstruction and Detector Software  Physics Analysis è These are closely related but are dealt with outside of this particular project as an “extended project” or as “related activities” Project Scope

October 23, 2000USCMS S&C Project Matthias Kasemann12 CAS Subproject In addition to developing software, this subproject will also provide expert programming personnel to assist the physicists in developing reconstruction and physics analysis programs by serving as mentors, reviewers, advisers and, where appropriate as software tool-writers. This will ensure that the software produced will be easy to integrate into the whole system, will be efficient in its use of hardware resources, and will be maintainable and adaptable for the full life of the CMS data analysis. è To support the design, development, modeling optimization and commissioning of software related to detectors being constructed by US CMS è To provide its share of the framework and infrastructure software required to support data analysis and simulation for CMS è For remote collaborative tools to enable the distributed model of computing that permits members of US CMS to carry out analysis whether they are at home in the US or visiting or resident at CERN è To satisfy any specialized needs required to carry out data analysis activities of interest to members of US CMS

October 23, 2000USCMS S&C Project Matthias Kasemann13 UF Subproject è The goal of the User Facilities Subproject is to provide the enabling infrastructure of software and computing that will allow US physicists to fully participate in the physics program of CMS. è To this end the subproject will acquire, develop, install, integrate, commission and operate the hardware and software for the facilities required to support the development and data analysis activities of USCMS. è This subproject will include a major ‘Tier1’ regional computing center at Fermilab to support US physicists working on CMS. It is appropriately sized to support this community which comprises 20% of the full CMS collaboration. è Tier 2 Centers are part of the User Facilities Subproject

October 23, 2000USCMS S&C Project Matthias Kasemann14 Tier 2 Centers  Tier 2s will partner with a Tier 1 which will provide support and liaison with CERN.  US CMS Tier 2 Centers (or the CMS parts of multiple customer centers) appear as Level 3 WBS items under the UF project, and the UF Level 2 manager has specific responsibilities with respect to them è Tier 1 interfaces to the Tier 2’s in the US è The Tier 2 CMS representative is a L3 manager of the US S&C project

October 23, 2000USCMS S&C Project Matthias Kasemann15 Organization of US CMS Projects

October 23, 2000USCMS S&C Project Matthias Kasemann16 Role of the ASCB  Helps in Development of the Project Plan  Advises on Scientific and Technical Policy  Advises on appointments  Provides Continuous Project Input and Feedback throughout the life of the construction project The US CMS Software and Computing Board provides crucial input and feedback for the US CMS Software and Computing project. It advises the Level 1 Project Manager and provides liaison to US CMS.

October 23, 2000USCMS S&C Project Matthias Kasemann17 Composition of ASCB  Specifies elected and Ex Officio members  six members elected at large from the US CMS collaboration;  the US CMS Physics Coordinator (also elected);  the Chair of the US CMS Collaboration Board (ex-officio)  the Head of the Fermilab Computing Division (ex-officio)  the CMS Project Manager for Core Software + Computing (ex- officio)  the Project Manager of the US CMS Construction Project (ex- officio).  Level 1 Project Manager of the US CMS Software and Computing Project (ex-officio)  Two Level 2 Project Managers of the US CMS Software and Computing Project (ex-officio) è The 7 elected members will choose a chairperson from among the 6 members who were elected at large è Term is 2 years, 3 positions elected every year

October 23, 2000USCMS S&C Project Matthias Kasemann18 Relation to CMS èTo create a strong linkage between the US CMS Software and Computing Project and the overall CMS Project, the CMS L2 manager will be an ex officio member of the USASCB. è The USASCB will have the responsibility for providing liaison between the US CMS Computing Project and the CMS Software and Computing Board. è The US CMS Software and Computing Project Level 1 Project Manager will act as liaison to the CMS Software and Computing Technical Board.

October 23, 2000USCMS S&C Project Matthias Kasemann19

October 23, 2000USCMS S&C Project Matthias Kasemann20 US-CMS Sub-System Links to International CMS Collaboration Board HCAL PM ECAL PM TRIDAS PM EMU PM Tracker PM Common Proj. PM HCAL PM ECAL PM TRIDAS PM Muon PM Tracker PM Note: Subsystem Institution and Finance Boards are omitted for clarity SW/Comp. PM Magnet PM Management Board Steering Committee US-CMSCMS

October 23, 2000USCMS S&C Project Matthias Kasemann21 “Extended” Lines of Communication CMS SCB CMS SCTB US S&C L1PM US ASCB p Collaboration-elected members, chair of US CMS CB, Physics Coordinator p Construction Project Technical Director p FNAL CD -- head of CD p Technical -- L1, L2 PMs p CMS -- S&C Project Manager Joint Oversight Group Fermilab Director or Designee Software and Computing Project Project Manager Project Management Group p Fermilab Director or Designee -- aka FNAL Deputy Director and Assoc. Director for Research p Technical -- L1, L2 PMs p Collaboration -- chair of USASCB and chair of US CMS CB p Construction Project Technical Director p Head of FNAL CD

October 23, 2000USCMS S&C Project Matthias Kasemann22 External Review Committee è The chair of the PMG will establish a standing external review committee that will periodically examine and evaluate all aspects of the US CMS Software and Computing Project.

October 23, 2000USCMS S&C Project Matthias Kasemann23 Formal Agreements with CMS  The US CMS collaboration will be contributing to CMS computing in a variety of ways, each of which will have an appropriate formal mechanism for establishing milestones, deliverables, and specifications.  Levels of support for production activities, including those required to support the design, testing, simulation, and commissioning of the detector should be supported by MOUs negotiated with CMS by the L1PM with input from the US ASCB and with the approval of the PMG and funding agencies.

October 23, 2000USCMS S&C Project Matthias Kasemann24 Formal Agreements with CMS (cont’d)  The software development that directly relates to the international CMS effort should be developed as part of the CMS software and computing plan and approved, presumably as part of the project plan for the US CMS Software and Computing Project, by the PMG and the funding agencies.  Software efforts specifically in support of US physicists or intended to solve particular problems specific to the US, should be developed as part of the project plan with substantial input from US ASCB and approved by the PMG and, if required, by the funding agencies.

October 23, 2000USCMS S&C Project Matthias Kasemann25 The Project Organization: Status Advisory Software and Computing Board (ASCB)  ChairI. Gaines (FNAL) for 1 year  Elected MembersP. Avery (Florida) for 2 years S. Dasu (Wisconsin) for 1 year S. Eno, (Maryland) for 1 year S. Kunori (Maryland) for 2 years D. Stickland (Princeton) for 2 years  Physics CoordinatorJ. Branson (UCSD)for 2 years  Chair of US CMS CB H. Newman (Caltech), ex-officio  Construction Project ManagerD. Green (FNAL), ex-officio  Int’l CMS S&C Coordinator M. Pimia (CERN)ex-officio  Head of FNAL/CD M. Kasemannex-officio  Acting L1 PMM. Kasemann ex-officio  Acting L2 PM – CASL. Taylor ex-officio  Acting L2 PM – UFV. O’Dell ex-officio Level 1 Project Manager L. Bauerdick (DESY/ZEUS -> FNAL) è Appointment Confirmed by JOG 6/00 è Starts November 2000

October 23, 2000USCMS S&C Project Matthias Kasemann26 US CMS S&C Subprojects: Status  Core Application Software Subproject u Resource-loaded WBS for CMS and US CMS u Task- Oriented Requirements + US Support u US part: 9 (2000) - 13 (2004) FTEs of software engineering  User Facilities Subproject u Implement R&D and Prototype Systems: r Preproduction ODBMS and Event-distribution systems r Simulated Event service for PRS and ORCA teams u Implement Production Systems in u Replenish and Upgrade from u Staff: u Tier 1: 35 FTEs by 2006, partly leveraged from FNAL/CD u Tier 2: 5  1.5 FTEs = 7.5 FTEs Caltech, NEU, Princeton, UC Davis + FNAL FNAL + Tier 2 sites

October 23, 2000USCMS S&C Project Matthias Kasemann27 US CMS S&C: Data Grid Deploy computing resources as hierarchical grid è Tier 0  Central laboratory computing resources (CERN) è Tier 1  National center (Fermilab / BNL) è Tier 2  Regional computing center (Universities) è Tier 3  University group computing resources è Tier 4  Individual workstation/CPU u “Data Grid” to reflect the predominant role of data in the distributed analysis; Concept developed by Caltech, U. Florida and FNAL u This concept and “Tiers” has been adopted throughout Europe by the EU Grid Project u We have to start implementation of Tier 1 and Tier2 R&D systems now to: è Support physics and HLT studies è Support software development è Perform R&D on distributed data analysis

October 23, 2000USCMS S&C Project Matthias Kasemann28 US CMS Tier2 planning  FY 2000/1Build basic services; HLT milestones 1-2 prototype-Tier 2Studies with ORCA  FY 2001Initial Grid systemFile replication service work with Tier 1 Multi-site cached file access  FY 2002Second set of Tier2 CMS Data Challenges centersS&C TDR and Physics TDR  FY 2003/4 Tier 2 centers 5%-Scale Data Challenge + at last set of sites PhysicsTDR; production data grid test  FY 2004/5 Production-quality 20% Production Grid System CMS Mock Data Challenge  FY 2005/6 Full distributed system Final Production Shakedown  R&D systems: leverage existing resources at Universities  Funding for Tier 2’s to come mostly from NSF initiatives

October 23, 2000USCMS S&C Project Matthias Kasemann29 US CMS Tier 2 Centers  Discussed at US CMS Collaboration Board, May 20, 2000 è Agreed: start deployment of 1 prototype Tier 2 center è Query sent to each US CMS institute: p Interest and plans to participate in Tier2 R&D now? p Who can work on it? è Result: p Ready to started “now”: Caltech + UC Davis + UCSD9/00 Florida, 4/01 Univ. of Iowa + Iowa State~4/01 p Start later for production:Boston, Maryland, Minnesota Wisconsin è Next steps: p Consolidate Tier2 plans for R&D on distributed data analysis possibility to contribute to Monte Carlo production Possible leverage from Universities

October 23, 2000USCMS S&C Project Matthias Kasemann30 First Prototype Tier 2 in California  Caltech + UC San Diego + UC Davis è Hardware Plan + Cost: 80 Dual CPU + Disk Linux Nodes$ 200 k [40 in Caltech, 40 in UCSD] Sun Data Server with RAID Array$ 30 k [partly from Caltech] Tape Library$ 20 k [in Caltech] LAN Switches$ 50 k Collaborative Infrastructure Upgrades $ 10 k Installation and Infrastructure$ 30 k Net Connect to Abilene$ 0 k Tape Media and Consumables$ 10 k Staff (Ops and System Support)$ 50 k Gigabit data access at UC Davis$ 30 k Total Estimated Cost (First Year)$ 430 k UCSD cost sharing $ -50 k UC Davis cost sharing$ -30 k Cost $ 350 k  Status: è Funding allocation request submitted to start first Tier 2 center in California è Hardware is arriving at sites

October 23, 2000USCMS S&C Project Matthias Kasemann31 PMP: Status/Summary - I  WBS exists for the two major subprojects  Resource estimates exist è Revised recently: p With input from Int’l CMS planning during Hoffmann Review at CERN p After first funding guidelines received from DOE in May  Milestones and timelines exist and are frequent and aggressive enough to drive the project and its oversight  Key management roles are identified è ASCB elected in September 2000 è L1 PM identified, starts November 2000  Key interfaces are worked out in a manner consistent with the collaborative nature of CMS and US CMS and with their “scientific culture”  Key interfaces are worked out with funding agencies and CMS  A draft of a full Project Management Plan exists è We separated the “stable” (project organization and oversight) aspects of the plan from the more “changeable” aspects (funding dependent). We created two documents: p Project Management and Work Breakdown Structure p Schedules, Milestones, and Budgets (finishing now)

October 23, 2000USCMS S&C Project Matthias Kasemann32 PMP: Status/Summary - II PMP: Status/Summary - II  The plan draws strength and exploits the synergy between p US Universities and Fermilab p Software Professionals and physicists  It takes advantage of and contributes to key developments in the US in information technology p Drive towards a high speed network infrastructure p Development of ever better network software and applications such as grid computing concepts  It takes advantage through the Tier2 centers of the significant strengths of US universities in the area of computer science and information technology These are key elements to its eventual success