1. Distributed IT Infrastructure for U.S. ATLAS Rob Gardner Indiana University DOE/NSF Review of U.S. ATLAS and CMS Computing Projects Brookhaven National Laboratory NOVEMBER 14-17, 2000

2. November 14-17, 2000 Rob Gardner Tier 2 Plan for U.S. ATLAS 2 Outline  Requirements  Approach  Organization  Resource Requirements  Schedule  Fallback Issues

3. November 14-17, 2000 Rob Gardner Tier 2 Plan for U.S. ATLAS 3 Distributed IT Infrastructure  A wide area computational infrastructure for U.S. ATLAS  A network of distributed computing devices  A network of distributed data caches & stores  Connectivity  Physicists with data (laptop scale sources: LOD)  Computers with data (at all scales)  Physicists with each other (collaboration)  Distributed information, portals  Efforts  Data Grid R&D  Strategic “remote” sites (Tier 2s)  Distributed IT support at the Tier 1 center

4. November 14-17, 2000 Rob Gardner Tier 2 Plan for U.S. ATLAS 4 Requirements  Access  Efficient access to resources at the Tier 1 facility  Data distribution to remote computing devices  Information  A secure infrastructure to locate, monitor and manage collections of distributed resources  Analysis planning framework  Resource estimation  “Matchmaker” tools to optimally connect physicist+CPU+data+etc…  Scalable  Add arbitrary large numbers of computing devices as they become available  Add arbitrarily large numbers of data sources as they become available

5. November 14-17, 2000 Rob Gardner Tier 2 Plan for U.S. ATLAS 5 Approach  ~5 strategic remote sites (Tier 2s)  Scale of each facility:  MONARC estimates  ATLAS NCB/WWC (World Wide Computing Group)  National Tier 1 facility  209 Spec95  365 Online disk  2 PB tertiary  Tier 2 = Tier 1 * 20%

6. November 14-17, 2000 Rob Gardner Tier 2 Plan for U.S. ATLAS 6 Role of Tier 2 Centers  User Analysis  Standard configuration optimized for analysis at the AOD level  ESD objects required for some analysis  Primary Resource for Monte Carlo Simulation  “Spontaneous” production level ESD skims (autonomy)  Data distribution caches  Remote data stores  HSM serve to archive AODs  MC data of all types (GEN, RAW, ESD, AOD, LOD) from all Tier 2’s & users

7. November 14-17, 2000 Rob Gardner Tier 2 Plan for U.S. ATLAS 7 Typical Tier 2  CPU: 50K SpecInt95 (t1: 209K)  Commodity Pentium/Linux  Estimated 144 Dual Processor Nodes (t1: 640)  Online Storage: 70 TB Disk (t1: 365)  High Performance Storage Area Network  Baseline: Fiber Channel Raid Array

8. November 14-17, 2000 Rob Gardner Tier 2 Plan for U.S. ATLAS 8 ‘Remote’ Data Stores  Exploit existing infrastructure  mass store infrastructure at 2 of the 5 Tier 2 centers  Assume existing HPSS or equivalent license, tape silo, robot  Augment with drives, media, mover nodes, and disk cache  Each site contributes 0.3-0.5 PB store  AOD archival, MC ESD+AOD archival

9. November 14-17, 2000 Rob Gardner Tier 2 Plan for U.S. ATLAS 9 Organization  Facilities Subproject 2.3.2

10. November 14-17, 2000 Rob Gardner Tier 2 Plan for U.S. ATLAS 10 Personnel MANPOWER ESTIMATE SUMMARY IN FTEs WBSNo:2Funding Type:Infrastructure11/13/00 8:08:38 PM Description:US ATLAS ComputingInstitutions:AllFunding Source :All FY 01FY 02FY 03FY 04FY 05FY 06Total IT I1.04.06.010.010.07.038.0 IT II.01.02.02.05.05.015.0 Physicist1.01.01.01.01.0.05.0 TOTAL LABOR 2.06.09.013.016.012.058.0

11. November 14-17, 2000 Rob Gardner Tier 2 Plan for U.S. ATLAS 11 Tier 2 Costs

12. November 14-17, 2000 Rob Gardner Tier 2 Plan for U.S. ATLAS 12 Schedule  R&D Tier 2’s – FY ‘01 & FY ‘02  Initial Development & Test, 1% to 2% scale  Start Grid testbed: ATLAS-GriPhyN  Data Challenges – FY ‘03 & FY ‘04  Production Tier 2’s – FY ‘04 & FY ‘05  Operation – FY ‘05, FY ‘06 & beyond  Full Scale System Operation, 20% (‘05) to 100% (‘06) (as for Tier 1)

13. November 14-17, 2000 Rob Gardner Tier 2 Plan for U.S. ATLAS 13 Calren Esnet, Abilene, Nton Abilene Esnet, Mren UC Berkeley LBNL-NERSC Esnet NPACI, Abilene Brookhaven National Laboratory Indiana University Boston University Argonne National Laboratory HPSS sites U Michigan University of Texas at Arlington Testbed ‘01

14. November 14-17, 2000 Rob Gardner Tier 2 Plan for U.S. ATLAS 14 Fallback Issues  Impact of limited support for a planned distributed infrastructure?  Several scenarios of course are possible  US ATLAS will face a serious shortfall in analysis capability  Shortfall in simulation capacity  Analysis groups will have less autonomy  University groups likely augment their facilities through supplemental requests, and large scale proposals to establish multidisciplinary “centers”  We could end up with 1 Tier 1 and 32 “Tier 2” centers  An incoherent, messy infrastructure, difficult to manage  Not the best way to optimize physics discovery