Ian Foster Argonne National Laboratory University of Chicago Univa Corporation Service-Oriented Science Scaling Science Services APAC Conference, September 28, 2005 iGrid Workshop, September 27, 2005

2 Two Questions l How do we scale the number of scientists benefiting from computational techniques? l What should be the role of infrastructure providers in enabling this scaling?

3 Computational Science  Increasingly sophisticated computational approaches DMonolithic programs, databases u Inflexible & hard to evolve u Mismatch with reality of diverse & distributed teams, resources, & approaches Computation joins theory & experiment as a third mode of scientific enquiry Genbank Program & data PC or Supercomputer

4 Decompose over the Network l Clients can then integrate dynamically u Select & compose services u Select “best of breed” providers u Publish result as a new service l Need not know implementation details l Note: complements, not replaces, HPC Service-Oriented Architecture

5 For Example: Virtual Observatories Surveys Observatories Missions Survey and Mission Archives Digital libraries Numerical Sim’s Sloan vs. 2MASS Brown dwarf candidates

6 Having Decomposed, Integrate l For example u Registries u Value-added services u Workflows l Issues u Description u Discovery u Composition u Adaptation & evolution u Qualities of service: security, performance, reliability, … Data Archives Analysis tools Discovery tools Users

7 Example Value Added Service: PUMA PUMA Knowledge Base Information about proteins analyzed against ~2 million gene sequences Analysis on Grid Involves millions of BLAST, BLOCKS, and other processes Natalia Maltsev et al.

8 SOA= Silo-Oriented Architecture? l What about dynamic behaviors? u Time-varying load u Dynamically instantiated services l What about operating costs? u Software deployment & maintenance u Security & other concerns Week Operating Services or?

9 We Need to Decompose in Two Dimensions Horizontal

10 We Need to Decompose in Two Dimensions Horizontal Vertical

11 IPC Dispatcher Globus Provision New Worker Process IPC Server 2 Decomposition Enables On-Demand Provisioning l Aggregate resources l Deliver to services l Separate production & consumption l Issues u Discovery u Composition u Qualities of service SAP GlobusWorld Demo IPC = Internet Pricing Configurator

12  Cardiff AEI/Golm Birmingham The Globus-Based LIGO Data Grid Replicating >1 Terabyte/day to 8 sites >30 million replicas so far MTBF = 1 month LIGO Gravitational Wave Observatory

13 Decomposition Enables Separation of Concerns & Roles User Service Provider “Provide access to data D at S1, S2, S3 with performance P” Resource Provider “Provide storage with performance P1, network with P2, …” D S1 S2 S3 D S1 S2 S3 Replica catalog, User-level multicast, … D S1 S2 S3

14 Scaling Up “Sometimes through heroism you can make something work. However, understanding why it worked, abstracting it, making it a primitive is the key to getting to the next order of magnitude of scale.” Robert Calderbank  We want to scale the number, robustness, & performance of services

15 Identifying Primitives: (1) Taking Services Seriously l Model the world as a collection of services u Computations, computers, instruments, storage, data, communities, agreements, … l Focus on what these things have in common l E.g., lifecycle management u Negotiation, deployment/creation, modeling, monitoring, management, termination l E.g., security u Authentication, authorization, audit, …  Web Services-based Grid infrastructure I. Foster, S. Tuecke, Describing the Elephant: The Many Faces of IT as Service, ACM Queue, 2005

16 Identifying Primitives: (2) Interface Specifications Web services (WSDL, SOAP, WS-Security, WS-ReliableMessaging, …) WS-Resource Framework & WS-Notification* (Resource identity, lifetime, inspection, subscription, …) WS-Agreement (Agreement negotiation) WS Distributed Management (Lifecycle, monitoring, …) Applications of the framework (Compute, network, storage provisioning, job reservation & submission, data management, application service QoS, …) *WS-Transfer, WS-Enumeration, WS-Eventing, WS-Management define similar functions Foster, Czajkowski, Frey, et al., From OGSI to WSRF, Proc. IEEE, 93(3)

17 Identifying Primitives: (3) Open Source Implementation Data Mgmt Security Common Runtime Execution Mgmt Info Services GridFTP Authentication Authorization Reliable File Transfer Data Access & Integration Grid Resource Allocation & Management Index Java Runtime Community Authorization Data Replication Community Scheduling Framework Delegation Replica Location Trigger C Runtime Python Runtime WebMDS Workspace Management Grid Telecontrol Protocol Credential Mgmt I. Foster, Globus Toolkit Version 4: Software for Service-Oriented Systems, LNCS 3779, 2-13, 2005

18 Jobs (2004) Open Science Grid  50 sites (15,000 CPUs) & growing  400 to >1000 concurrent jobs  Many applications + CS experiments; includes long-running production operations  Up since October 2003; few FTEs central ops

19 Virtual OSG Clusters OSG cluster Xen hypervisors TeraGrid cluster OSG

20 Dynamic Service Deployment Community A Community Z … Community scheduling logic Data distribution Community management Science services... Requirements: Community control Persistence Resource guarantees Non- interference

21 Summary l How do we scale the number of scientists benefiting from computational techniques?  Construct powerful science services  Simplify construction by decomposing roles: content, function, resource l What should be the role of infrastructure providers in enabling this scaling?  Service providers for communities wanting to deliver content  Resource providers for service providers wanting to deliver services

22 Domain-independentDomain-dependent Content Function Resources Experimental apparatus Servers, storage, networks Metadata catalog Data archive Simulation server Certificate authority Simulation code Expt design Telepresence monitor Simulation code Expt output Electronic notebook Portal server Service-Oriented Science: Scaling by Separating Concerns Hosted by Enabled by I. Foster, Service-Oriented Science, Science, 308, May 6, 2005

23 Acknowledgments l NSF, DOE, NASA, IBM for financial support l Numerous fine colleagues at Argonne, U.Chicago, USC/ISI, and elsewhere l In particular: Carl Kesselman Steve Tuecke Kate Keahey & Bill Allcock, Ann Chervenak, Ewa Deelman, Jennifer Schopf, Mike Wilde

24 For More Information l Globus Alliance: l Papers: For those at IGrid: Carl Kesselman’s Master Class (Thursday) For those at APAC: Globus Toolkit Tutorial (Thursday, Friday)