The Grid as Future Scientific Infrastructure Ian Foster Argonne National Laboratory University of Chicago Globus Alliance

2 ARGONNE  CHICAGO Overview l Evolution of scientific infrastructure u Technology drivers & new requirements l Building cyberinfrastructure u Standards, software, and facilities u Examples l Futures u Virtual data u Discipline-specific cyberinfrastructures u Challenges

3 ARGONNE  CHICAGO Technology Drivers l Internet revolution: 100M+ hosts u Collaboration & sharing the norm l Universal Moore’s law: x10 3 /10 yrs u Sensors as well as computers l Petascale data tsunami u Gating step is analysis l & our old infrastructure? 114 genomes 735 in progress You are here

4 ARGONNE  CHICAGO If We Can’t Do it All in our Lab … l Then we need to be able to reach out into the network to obtain needed resources u Clusters, archives, instrumentation l And integrate those resources & services into our data processing workflows u Operating at scale (not Web browsing) l Which means we need u Community facilities u Standard means of accessing those facilities u Means of coordinating across facilities u New methodologies and tools

5 ARGONNE  CHICAGO Earth Simulator Atmospheric Chemistry Group LHC Exp. Astronomy Grav. Wave Nuclear Exp. Current accelerator Exp. Scale Metrics: Participants, Data, Tasks, Performance, Interactions, …

6 ARGONNE  CHICAGO l Resources u Computing, storage, data l Communities u Operational procedures, … Cyberinfrastructure—AKA Grid A A A l Services u Authentication, discovery, … l Connectivity u Reduce tyranny of distance l Technologies u Build services & applications

7 ARGONNE  CHICAGO Overview l Evolution of scientific infrastructure u Technology drivers & new requirements l Building cyberinfrastructure u Standards, software, and facilities u Examples l Futures u Virtual data u Discipline-specific cyberinfrastructures u Challenges

8 ARGONNE  CHICAGO Building Cyberinfrastructure l Open standards: absolutely vital u Define how we describe, discover, access, secure, monitor, & manage services u E.g., Open Grid Services Architecture l Open facilities: enabled by standards u Standards-based access, community services (authentication, registry), operational support u E.g., TeraGrid, Grid3, NEESgrid, ESG, DOE Science Grid, LHC Computing Grid, NASA IPG l Open software: accelerate standards adoption u Accelerates creation of facilities & applications u Globus Toolkit, NSF Middleware Initiative, etc.

9 ARGONNE  CHICAGO Increased functionality, standardization Custom solutions Open Grid Services Arch Real standards Multiple implementations Web services, etc. Managed shared virtual systems Computer science research Globus Toolkit Defacto standard Single implementation Internet standards The Emergence of Open Grid Standards 2010

10 ARGONNE  CHICAGO Grid-Web Services Convergence Web Services Messaging, Security, Etc. Open Grid Services Infrastructure Domain-Specific Services Core Services Program Execution Data Services

11 ARGONNE  CHICAGO Grid-Web Services Convergence Completed: A Major Milestone! Web Services Messaging, Security, Etc. Open Grid Services Infrastructure Domain-Specific Services Core Services Program Execution Data Services WS-Resource Framework

Deploying Cyberinfrastructure

13 ARGONNE  CHICAGO

14 ARGONNE  CHICAGO NEESgrid Earthquake Engineering Collaboratory U.Nevada Reno

15 ARGONNE  CHICAGO NEESgrid Multisite Online Simulation Test (July 2003) Illinois Colorado Illinois (simulation) W

16 ARGONNE  CHICAGO Earth System Grid (ESG) Goal: address technical obstacles to the sharing & analysis of high-volume data from advanced earth system models

17 ARGONNE  CHICAGO Grid2003: Towards a Persistent U.S. Open Science Grid Status on 11/19/03 ( P

18 ARGONNE  CHICAGO Virtual Observatories No. & sizes of data sets as of mid-2002, grouped by wavelength 12 waveband coverage of large areas of the sky Total about 200 TB data Doubling every 12 months Largest catalogues near 1B objects Data and images courtesy Alex Szalay, John Hopkins

19 ARGONNE  CHICAGO Resource Center (Processors, disks) Grid server Nodes Resource Center Resource Center Resource Center Operations Center Regional Support Center (Support for Applications Local Resources) Regional Support Regional Support Regional Support EGEE: Enabling Grids for E-Science in Europe

20 ARGONNE  CHICAGO Overview l Evolution of scientific infrastructure u Technology drivers & new requirements l Building cyberinfrastructure u Standards, software, and facilities u Examples l Futures u Virtual data u Discipline-specific cyberinfrastructures u Challenges

21 ARGONNE  CHICAGO Science as Workflow: Integrating & Evolving Data and Computation “Science advances through tentative answers to a series of more & more subtle questions which reach deeper & deeper into the essence of natural phenomena.” (L. Pasteur) l In a networked petascale world, this means leveraging distributed data & computation to u Discover relevant data/hypotheses u Derive new tentative data/hypotheses from old u Publish new data/hypotheses for use by others l In a collaborative framework in which many can contribute to, & draw from, the whole

Integrating Across Different Types of Information ID MURA_BACSU STANDARD; PRT; 429 AA. DE PROBABLE UDP-N-ACETYLGLUCOSAMINE 1-CARBOXYVINYLTRANSFERASE DE (EC ) (ENOYLPYRUVATE TRANSFERASE) (UDP-N-ACETYLGLUCOSAMINE DE ENOLPYRUVYL TRANSFERASE) (EPT). GN MURA OR MURZ. OS BACILLUS SUBTILIS. OC BACTERIA; FIRMICUTES; BACILLUS/CLOSTRIDIUM GROUP; BACILLACEAE; OC BACILLUS. KW PEPTIDOGLYCAN SYNTHESIS; CELL WALL; TRANSFERASE. FT ACT_SITE BINDS PEP (BY SIMILARITY). FT CONFLICT S -> A (IN REF. 3). SQ SEQUENCE 429 AA; MW; 02018C5C CRC32; MEKLNIAGGD SLNGTVHISG AKNSAVALIP ATILANSEVT IEGLPEISDI ETLRDLLKEI GGNVHFENGE MVVDPTSMIS MPLPNGKVKK LRASYYLMGA MLGRFKQAVI GLPGGCHLGP RPIDQHIKGF EALGAEVTNE QGAIYLRAER LRGARIYLDV VSVGATINIM LAAVLAEGKT IIENAAKEPE IIDVATLLTS MGAKIKGAGT NVIRIDGVKE LHGCKHTIIP DRIEAGTFMI [source: GlaxoSmithKline via Carole Goble]

Virtual Data Research: Sharing Recipes As Well as Data

24 ARGONNE  CHICAGO Discipline Cyberinfrastructure: E.g., Life Sciences l Leverage emerging “eScience infrastructure” u Storage, data archives, networks, cluster l Create an extensible service-rich environment u Workflows (standard protocols) u Experimental facilities: MRI, EM, arrays, … u Data update and access services u Computational services: BLAST, etc., etc. u Tools (informatics) suites u Ontologies & conceptual discovery frameworks l Bio-applns-neutral: genomics, proteomics, …

25 ARGONNE  CHICAGO Other Observations l Building & applying cyberinstructure is hard u Need sustained effort and commitment l Not clear that current funding models apply u Must support a critical mass of experts to build & operate infrastructure u Software is a vital part of the whole l Discipline sciences must obtain experience u Start multidisciplinary exploratory projects l International cooperation is vital u Our partners “get it” and are hard at work

26 ARGONNE  CHICAGO For More Information l The Globus Alliance u l Global Grid Forum u l GriPhyN u l Background information u l GlobusWorld, Feb 2005 u 2nd Edition: November 2003