1. The Grid as Future Scientific Infrastructure Ian Foster Argonne National Laboratory University of Chicago Globus Alliance www.mcs.anl.gov/~foster

2. 2 foster@mcs.anl.gov ARGONNE  CHICAGO Overview l Evolution of scientific infrastructure u Technology drivers & new requirements l Building cyberinfrastructure u Standards, software, and facilities u Examples l Futures u Virtual data u Discipline-specific cyberinfrastructures u Challenges

3. 3 foster@mcs.anl.gov ARGONNE  CHICAGO Technology Drivers l Internet revolution: 100M+ hosts u Collaboration & sharing the norm l Universal Moore’s law: x10 3 /10 yrs u Sensors as well as computers l Petascale data tsunami u Gating step is analysis l & our old infrastructure? 114 genomes 735 in progress You are here

4. 4 foster@mcs.anl.gov ARGONNE  CHICAGO If We Can’t Do it All in our Lab … l Then we need to be able to reach out into the network to obtain needed resources u Clusters, archives, instrumentation l And integrate those resources & services into our data processing workflows u Operating at scale (not Web browsing) l Which means we need u Community facilities u Standard means of accessing those facilities u Means of coordinating across facilities u New methodologies and tools

5. 5 foster@mcs.anl.gov ARGONNE  CHICAGO Earth Simulator Atmospheric Chemistry Group LHC Exp. Astronomy Grav. Wave Nuclear Exp. Current accelerator Exp. Scale Metrics: Participants, Data, Tasks, Performance, Interactions, …

6. 6 foster@mcs.anl.gov ARGONNE  CHICAGO l Resources u Computing, storage, data l Communities u Operational procedures, … Cyberinfrastructure—AKA Grid A A A l Services u Authentication, discovery, … l Connectivity u Reduce tyranny of distance l Technologies u Build services & applications

7. 7 foster@mcs.anl.gov ARGONNE  CHICAGO Overview l Evolution of scientific infrastructure u Technology drivers & new requirements l Building cyberinfrastructure u Standards, software, and facilities u Examples l Futures u Virtual data u Discipline-specific cyberinfrastructures u Challenges

8. 8 foster@mcs.anl.gov ARGONNE  CHICAGO Building Cyberinfrastructure l Open standards: absolutely vital u Define how we describe, discover, access, secure, monitor, & manage services u E.g., Open Grid Services Architecture l Open facilities: enabled by standards u Standards-based access, community services (authentication, registry), operational support u E.g., TeraGrid, Grid3, NEESgrid, ESG, DOE Science Grid, LHC Computing Grid, NASA IPG l Open software: accelerate standards adoption u Accelerates creation of facilities & applications u Globus Toolkit, NSF Middleware Initiative, etc.

9. 9 foster@mcs.anl.gov ARGONNE  CHICAGO Increased functionality, standardization Custom solutions 1990199520002005 Open Grid Services Arch Real standards Multiple implementations Web services, etc. Managed shared virtual systems Computer science research Globus Toolkit Defacto standard Single implementation Internet standards The Emergence of Open Grid Standards 2010

10. 10 foster@mcs.anl.gov ARGONNE  CHICAGO Grid-Web Services Convergence Web Services Messaging, Security, Etc. Open Grid Services Infrastructure Domain-Specific Services Core Services Program Execution Data Services

11. 11 foster@mcs.anl.gov ARGONNE  CHICAGO Grid-Web Services Convergence Completed: A Major Milestone! Web Services Messaging, Security, Etc. Open Grid Services Infrastructure Domain-Specific Services Core Services Program Execution Data Services WS-Resource Framework

12. Deploying Cyberinfrastructure

13. 13 foster@mcs.anl.gov ARGONNE  CHICAGO

14. 14 foster@mcs.anl.gov ARGONNE  CHICAGO NEESgrid Earthquake Engineering Collaboratory U.Nevada Reno www.neesgrid.org

15. 15 foster@mcs.anl.gov ARGONNE  CHICAGO NEESgrid Multisite Online Simulation Test (July 2003) Illinois Colorado Illinois (simulation) W

16. 16 foster@mcs.anl.gov ARGONNE  CHICAGO Earth System Grid (ESG) Goal: address technical obstacles to the sharing & analysis of high-volume data from advanced earth system models

17. 17 foster@mcs.anl.gov ARGONNE  CHICAGO Grid2003: Towards a Persistent U.S. Open Science Grid Status on 11/19/03 (http://www.ivdgl.org/grid2003) P

18. 18 foster@mcs.anl.gov ARGONNE  CHICAGO Virtual Observatories No. & sizes of data sets as of mid-2002, grouped by wavelength 12 waveband coverage of large areas of the sky Total about 200 TB data Doubling every 12 months Largest catalogues near 1B objects Data and images courtesy Alex Szalay, John Hopkins

19. 19 foster@mcs.anl.gov ARGONNE  CHICAGO Resource Center (Processors, disks) Grid server Nodes Resource Center Resource Center Resource Center Operations Center Regional Support Center (Support for Applications Local Resources) Regional Support Regional Support Regional Support EGEE: Enabling Grids for E-Science in Europe

20. 20 foster@mcs.anl.gov ARGONNE  CHICAGO Overview l Evolution of scientific infrastructure u Technology drivers & new requirements l Building cyberinfrastructure u Standards, software, and facilities u Examples l Futures u Virtual data u Discipline-specific cyberinfrastructures u Challenges

21. 21 foster@mcs.anl.gov ARGONNE  CHICAGO Science as Workflow: Integrating & Evolving Data and Computation “Science advances through tentative answers to a series of more & more subtle questions which reach deeper & deeper into the essence of natural phenomena.” (L. Pasteur) l In a networked petascale world, this means leveraging distributed data & computation to u Discover relevant data/hypotheses u Derive new tentative data/hypotheses from old u Publish new data/hypotheses for use by others l In a collaborative framework in which many can contribute to, & draw from, the whole

22. Integrating Across Different Types of Information ID MURA_BACSU STANDARD; PRT; 429 AA. DE PROBABLE UDP-N-ACETYLGLUCOSAMINE 1-CARBOXYVINYLTRANSFERASE DE (EC 2.5.1.7) (ENOYLPYRUVATE TRANSFERASE) (UDP-N-ACETYLGLUCOSAMINE DE ENOLPYRUVYL TRANSFERASE) (EPT). GN MURA OR MURZ. OS BACILLUS SUBTILIS. OC BACTERIA; FIRMICUTES; BACILLUS/CLOSTRIDIUM GROUP; BACILLACEAE; OC BACILLUS. KW PEPTIDOGLYCAN SYNTHESIS; CELL WALL; TRANSFERASE. FT ACT_SITE 116 116 BINDS PEP (BY SIMILARITY). FT CONFLICT 374 374 S -> A (IN REF. 3). SQ SEQUENCE 429 AA; 46016 MW; 02018C5C CRC32; MEKLNIAGGD SLNGTVHISG AKNSAVALIP ATILANSEVT IEGLPEISDI ETLRDLLKEI GGNVHFENGE MVVDPTSMIS MPLPNGKVKK LRASYYLMGA MLGRFKQAVI GLPGGCHLGP RPIDQHIKGF EALGAEVTNE QGAIYLRAER LRGARIYLDV VSVGATINIM LAAVLAEGKT IIENAAKEPE IIDVATLLTS MGAKIKGAGT NVIRIDGVKE LHGCKHTIIP DRIEAGTFMI [source: GlaxoSmithKline via Carole Goble]

23. Virtual Data Research: Sharing Recipes As Well as Data

24. 24 foster@mcs.anl.gov ARGONNE  CHICAGO Discipline Cyberinfrastructure: E.g., Life Sciences l Leverage emerging “eScience infrastructure” u Storage, data archives, networks, cluster l Create an extensible service-rich environment u Workflows (standard protocols) u Experimental facilities: MRI, EM, arrays, … u Data update and access services u Computational services: BLAST, etc., etc. u Tools (informatics) suites u Ontologies & conceptual discovery frameworks l Bio-applns-neutral: genomics, proteomics, …

25. 25 foster@mcs.anl.gov ARGONNE  CHICAGO Other Observations l Building & applying cyberinstructure is hard u Need sustained effort and commitment l Not clear that current funding models apply u Must support a critical mass of experts to build & operate infrastructure u Software is a vital part of the whole l Discipline sciences must obtain experience u Start multidisciplinary exploratory projects l International cooperation is vital u Our partners “get it” and are hard at work

26. 26 foster@mcs.anl.gov ARGONNE  CHICAGO For More Information l The Globus Alliance u www.globus.org l Global Grid Forum u www.ggf.org l GriPhyN u www.griphyn.org l Background information u www.mcs.anl.gov/~foster l GlobusWorld, Feb 2005 u www.globusworld.org 2nd Edition: November 2003