1. HEP-CCC Meeting, November 1999Grid Computing for HEP L. E. Price, ANL Grid Computing for HEP L. E. Price Argonne National Laboratory HEP-CCC Meeting CERN, November 12, 1999

2. HEP-CCC Meeting, November 1999Grid Computing for HEP L. E. Price, ANL The Challenge Providing rapid access to event samples and subsets from massive datastores, from 100s of Terabytes in 2000 to 100 Petabytes by 2010. Transparent access to computing resources, throughout the U.S., and throughout the World The extraction of small or subtle new physics signals from large and potentially overwhelming backgrounds Enabling access to the data, and to the rest of the physics community, across and ensemble of networks of varying capability and reliability, using heterogeneous computing resources

3. HEP-CCC Meeting, November 1999Grid Computing for HEP L. E. Price, ANL Achieving a Balance Proximity of the data to central computing and data handling resources Proximity of frequently accessed data to the users, to be processed in desktops, local facilities, or regional centers Making efficient use of limited network bandwidth; especially transoceanic Making appropriate use of regional and local computing and data handling Involving scientists and students in each world region in the physics analysis

4. HEP-CCC Meeting, November 1999Grid Computing for HEP L. E. Price, ANL Need for optimization Meeting the demands of hundreds of users who need transparent access to local and remote data in disk caches and tape stores Prioritizing hundreds to thousands of requests from the local and remote communities Structuring and organizing the data; providing the tools for locating, moving, and scheduling data transport between tape and disk and across networks Ensuring that the overall system is dimensioned correctly to meet the aggregate need

5. HEP-CCC Meeting, November 1999Grid Computing for HEP L. E. Price, ANL Science and Massive Datasets Massive dataset generation the new norm in science –High Energy Physics –Nuclear Physics –LIGO –Automated astronomical scans (e.g., Sloan Digital Sky Survey) –The Earth Observing System (EOS) –The Earth System Grid –Geophysical data (e.g., seismic) –Satellite weather image analysis –The Human Brain Project (time series of 3-D images) –Protein Data Bank –The Human Genome Project –Molecular structure crystallography data

6. HEP-CCC Meeting, November 1999Grid Computing for HEP L. E. Price, ANL Proposed Solution A data analysis grid for High Energy Physics Tier 1 T2 3 3 3 3 3 3 3 3 3 3 3 3 CERN T2 4444 3 3

7. HEP-CCC Meeting, November 1999Grid Computing for HEP L. E. Price, ANL Analogy to Computing Grid Because the resources needed to solve complex problems are rarely collocated Topic of intensive CS research for a number of years already Computing (or data) resources from a “plug on the wall”

8. HEP-CCC Meeting, November 1999Grid Computing for HEP L. E. Price, ANL Why a Hierarchical Data Grid? Physical –Appropriate resource use  data proximity to users & labs –Efficient network use  local > regional > national > oceanic –Scalable growth  avoid bottlenecks Human –Central lab cannot manage / help / care about 1000s of users –Cleanly separates functionality of different resource types –University/regional computing complements national labs  funding agencies –Easier to leverage resources, maintain control, assert priorities at regional/local level –Effective involvement of scientists and students independently of location

9. HEP-CCC Meeting, November 1999Grid Computing for HEP L. E. Price, ANL Logical Steps toward Data Grid 2000200520101995 Production Basic Research Testbeds Design/Optimization (Pre)                             

10. HEP-CCC Meeting, November 1999Grid Computing for HEP L. E. Price, ANL U.S. Grid Technology Projects 2000200520101995 LHC, GriPhyN Clipper/ NGI-PPDG                           Apogee PASS/Globus/HENP-GC /MONARC/GIOD/Nile

11. HEP-CCC Meeting, November 1999Grid Computing for HEP L. E. Price, ANL In Progress Laboratory and experiment-specific development, deployment and operation (hardware and software); Tool development in HENP, Computer Science, Industry; The Particle Physics Data Grid: –NGI-funded project aiming (initially) at jump- starting the exploitation of CS and HENP software components to make major improvements in data access. Business as usual

12. HEP-CCC Meeting, November 1999Grid Computing for HEP L. E. Price, ANL Proposals being Developed GriPhyN: Grid Physics Networking –Targeted at NSF; –Focus on the long-term university-based grid infrastructure for major physics and astronomy experiments. APOGEE: A Physics-Optimized Grid Environment for Experiments –Targeted at DoE HENP (and/or DoE SSI); –Focus on medium to long-term software needs for HENP distributed data management; –Initial focus on instrumentation, modeling and optimization.

13. HEP-CCC Meeting, November 1999Grid Computing for HEP L. E. Price, ANL PPDG, APOGEE and GriPhyN A coherent program of work; Substantial common management proposed; A focus for HENP collaboration with Computer Science and Industry; PPDG/Apogee will create “middleware” needed by data-intensive science including LHC. (Synergy but no overlap with CMS/Atlas planning.)

14. HEP-CCC Meeting, November 1999Grid Computing for HEP L. E. Price, ANL Data Grid Projects in Context Construction and Operation of HENP Data Management an Analysis Systems Tiers 0/1 >> $20M/yr of existing funding at HENP labs. e.g. SLAC FY1999 ~$7M equipment for BaBar (of which < $2M physics CPU); ~$3M labor, M&S.

15. HEP-CCC Meeting, November 1999Grid Computing for HEP L. E. Price, ANL Data Grid Projects in Context Construction and Operation of HENP Data Management and Data Analysis Systems at DoE Laboratories Tiers 0/1 GriPhyN HENP Data Manage- ment at Major University Centers Tier 2 Draft proposal for NSF funding: $5-$16M/year $16M = $8M hardware $5M labor/R&D $3M network

16. HEP-CCC Meeting, November 1999Grid Computing for HEP L. E. Price, ANL Data Grid Projects in Context Construction and Operation of HENP Data Management and Data Analysis Systems at DoE Laboratories Tiers 0/1 GriPhyN HENP Data Manage- ment at Major University Centers Tier 2 OO Databases and Analysis Tools Resource Management Tools Metadata Catalogs WAN Data Movers Mass Storage Management Systems Matchmaking Widely Applicable Technolgy and Computer Science (not only from HENP; 100s of non-HEP FTEs)

17. HEP-CCC Meeting, November 1999Grid Computing for HEP L. E. Price, ANL Data Grid Projects in Context Construction and Operation of HENP Data Management and Data Analysis Systems at DoE Laboratories Tiers 0/1 GriPhyN HENP Data Manage- ment at Major University Centers Tier 2 OO Databases and Analysis Tools Resource Management Tools Metadata Catalogs WAN Data Movers Mass Storage Management Systems Matchmaking PPDG Particle Physics Data Grid NGI Project Large-scale tests/service focused on use of existing components

18. HEP-CCC Meeting, November 1999Grid Computing for HEP L. E. Price, ANL Data Grid Projects in Context Construction and Operation of HENP Data Management and Data Analysis Systems at DoE Laboratories Tiers 0/1 GriPhyN HENP Data Manage- ment at Major University Centers Tier 2 OO Databases and Analysis Tools Resource Management Tools Metadata Catalogs WAN Data Movers Mass Storage Management Systems Matchmaking PPDG Particle Physics Data Grid NGI Project Unified Project Management Optimization and Evaluation Instrumentation Modeling and Simulation A new level of rigor as the foundation for future progress APOGEE

19. HEP-CCC Meeting, November 1999Grid Computing for HEP L. E. Price, ANL Data Grid Projects in Context Construction and Operation of HENP Data Management and Data Analysis Systems at DoE Laboratories Tiers 0/1 GriPhyN HENP Data Manage- ment at Major University Centers Tier 2 OO Databases and Analysis Tools Resource Management Tools Metadata Catalogs WAN Data Movers Mass Storage Management Systems Matchmaking PPDG Particle Physics Data Grid NGI Project Unified Project Management Optimization and Evaluation Instrumentation Modeling and Simulation APOGEE R&D + Contacts with CS/Industry  Long-term Goals Testbeds

20. HEP-CCC Meeting, November 1999Grid Computing for HEP L. E. Price, ANL Overall Program Goal A Coordinated Approach to the Design and Optimization of a Data Analysis Grid for HENP Experiments

21. HEP-CCC Meeting, November 1999Grid Computing for HEP L. E. Price, ANL Particle Physics Data Grid Universities, DoE Accelerator Labs, DoE Computer Science Funded by DoE-NGI at $1.2M for first year

22. HEP-CCC Meeting, November 1999Grid Computing for HEP L. E. Price, ANL PPDG Collaborators Particle Accelerator Computer Physics Laboratory Science ANLXX LBNL XX BNLXX x CaltechXX FermilabXX x Jefferson LabXX x SLACXX x SDSCX WisconsinX

23. HEP-CCC Meeting, November 1999Grid Computing for HEP L. E. Price, ANL First Year PPDG Deliverables Implement and Run two services in support of the major physics experiments at BNL, FNAL, JLAB, SLAC: –“High-Speed Site-to-Site File Replication Service”; Data replication up to 100 Mbytes/s –“Multi-Site Cached File Access Service”: Based on deployment of file-cataloging, and transparent cache-management and data movement middleware First Year: Optimized cached read access to file in the range of 1-10 Gbytes, from a total data set of order One Petabyte Using middleware components already developed by the Proponents

24. HEP-CCC Meeting, November 1999Grid Computing for HEP L. E. Price, ANL PPDG Site-to-Site Replication Service u Network Protocols Tuned for High Throughput u Use of DiffServ for (1) Predictable high priority delivery of high - bandwidth data streams (2) Reliable background transfers u Use of integrated instrumentation to detect/diagnose/correct problems in long-lived high speed transfers [NetLogger + DoE/NGI developments] u Coordinated reservation/allocation techniques for storage-to-storage performance SECONDARY SITE CPU, Disk, Tape Robot PRIMARY SITE Data Acquisition, CPU, Disk, Tape Robot

25. HEP-CCC Meeting, November 1999Grid Computing for HEP L. E. Price, ANL PPDG Multi-site Cached File Access System University CPU, Disk, Users PRIMARY SITE Data Acquisition, Tape, CPU, Disk, Robot Satellite Site Tape, CPU, Disk, Robot Satellite Site Tape, CPU, Disk, Robot University CPU, Disk, Users University Users Satellite Site Tape, CPU, Disk, Robot

26. HEP-CCC Meeting, November 1999Grid Computing for HEP L. E. Price, ANL PPDG Middleware Components

27. HEP-CCC Meeting, November 1999Grid Computing for HEP L. E. Price, ANL APOGEE Focus on Instrumentation and Modeling Planned proposal to DOE Originally targeted at SSI Roughly the same collaborators as PPDG Intended to be the next step after PPDG

28. HEP-CCC Meeting, November 1999Grid Computing for HEP L. E. Price, ANL

29. HEP-CCC Meeting, November 1999Grid Computing for HEP L. E. Price, ANL Understanding Complex Systems (Writing into the BaBar Object Database at SLAC) Aug. 1: ~4.7 Mbytes/s Oct. 1: ~28 Mbytes/s

30. HEP-CCC Meeting, November 1999Grid Computing for HEP L. E. Price, ANL APOGEE Manpower Requirements (FTE) FY00FY01FY02FY03FY04 Instrumentation Low-level data capture0.510.750.750.75 Filtering and collecting agents0.51111 Data analysis and presentation0.5110.750.75 HENP workload profiling0.510.50.50.5 Simulation Framework design and development121.510.5 User workload simulation0.510.750.750.5 Component simulations (network, mass-storage system, object DB etc.)1.252.521.51 Site simulation packages111 Instrumentation/Simulation Testbed Instrumentation of existing experiment(s) (e.g.PPDG)0.51111 Acquire and simulate performance measurements0.250.50.50.751 Acquire user workload profile0.250.50.50.250.25 Test prediction and optimization0.50.750.75 Evaluation and Optimization Quantify evolving needs of physics (including site policies etc.)0.250.50.50.50.5 Develop metrics for usefulness of data management facilities0.51111 Optimize model systems0.511.5 Long-Term Strategy (Towards "Virtual Data") Tracking and testing HENP/CS/Industry developments121.51.51.5 Development projects in collaboration with HENP/CS/Industry 0.511.5 Project Management (APOGEE and PPDG) Project leader (physicist)0.51111 Lead computer scientist0.51111 TOTALS8.517171717

31. HEP-CCC Meeting, November 1999Grid Computing for HEP L. E. Price, ANL APOGEE Funding Needs $k$k$k$k$k FY00FY01FY02FY03FY04 Manpower Instrumentation250500406375375 Simulation344688656531375 Instrumentation/Simulation Testbed125250313344375 Evaluation and Optimization94188250313375 Long-Term Strategy (Towards "Virtual Data")125250250313375 Project Management (APOGEE and PPDG)225450450450450 Commercial Software100250375500500 Testbed hardware (in addition to parasitic 150400400400400 use of production systems) Workstations, M&S, Travel128255255255255 TOTALS15403230335534803480

32. HEP-CCC Meeting, November 1999Grid Computing for HEP L. E. Price, ANL GriPhyN Proposal Addresses several massive dataset problems  ATLAS, CMS  LIGO  Sloan Digital Sky Survey (SDSS)  Tier 2 computing centers (university based)  Hardware  commodity CPU / disk / tape  System support Networking Transatlantic link to CERN  "high-speed" Tier 2 backbone  multi-gigabit/sec R&D  Leverage Tier 2 + existing resources into Grid  Computer Science partnership, software

33. HEP-CCC Meeting, November 1999Grid Computing for HEP L. E. Price, ANL GriPhyN Goals Build production grid Exploit all computing resources most effectively Enable US physicists to participate fully in LHC program (also LIGO, SDSS)  Eliminate disadvantage of not being at CERN  Early physics analysis at LHC startup  Maintain and extend US leadership Build collaborative infrastructure for students & faculty  Training ground for next generation leaders

34. HEP-CCC Meeting, November 1999Grid Computing for HEP L. E. Price, ANL Tier 2 Regional Centers Total number »20  ATLAS:6  CMS:6  LIGO:5  SDSS2  3 Flexible architecture and mission  complements national labs  Intermediate-level data handling  Makes possible regional collaborations  Well-suited to universities (training, mentoring and education) Scale: Tier2 = (university * laboratory) 1/2  1 scenario:  Tier 2 = Tier 1  Tier 2  20% Tier 1

35. HEP-CCC Meeting, November 1999Grid Computing for HEP L. E. Price, ANL GriPhyN Funding (Very Rough)

36. HEP-CCC Meeting, November 1999Grid Computing for HEP L. E. Price, ANL R&D Proposal $15M (Jan. 1999) R&D goals (complementary to APOGEE / PPDG)  Data, resource management over wide area  Fault-tolerant distributed computing over LAN  High-speed networks, as they relate to data management  Grid testbeds (with end-users)  Simulations crucial to success  MONARC group  With APOGEE / PPDG  Leverage resources available to us  Strong connections with Computer Science people  Existing R&D projects  Commercial connections

37. HEP-CCC Meeting, November 1999Grid Computing for HEP L. E. Price, ANL Grid Computing: Conclusions HENP at the frontier of Information Technology –Collaboration with Computer Science; –Collaboration with industry; –Outreach to other sciences. –Visibility (and scrutiny) of HENP computing; Enabling revolutionary advances in data analysis in the LHC era –Increasing the value of the vital investment in experiment-specific data-analysis software