1. The Challenges of Grid Computing Ian Foster Mathematics and Computer Science Division Argonne National Laboratory and Department of Computer Science The University of Chicago http://www.mcs.anl.gov/~foster Condor Week Presentation at via Access Grid, March 4, 2002

2. foster@mcs.anl.gov ARGONNE  CHICAGO Trends in Grid Computing l New application domains –Data Grids, collaboratories l Integration with commercial technologies –Web services,.NET, …, … l Commercial acceptance and adoption –Very rapid; we don’t know what has hit us l Increasing scale –Sensor nets, mobile and wireless devices, Internet-wide deployment

3. foster@mcs.anl.gov ARGONNE  CHICAGO The Grid World: Current Status l Dozens of major Grid projects in scientific & technical computing/research & education l Considerable consensus on key concepts and technologies –Open source Globus Toolkit™ a de facto standard for major protocols & services –Far from complete or perfect, but out there, evolving rapidly, and large tool/user base l Industrial interest emerging rapidly l Opportunity: convergence of eScience and eBusiness requirements & technologies

4. foster@mcs.anl.gov ARGONNE  CHICAGO Grid Computing

5. foster@mcs.anl.gov ARGONNE  CHICAGO The Grid Problem Resource sharing & coordinated problem solving in dynamic, multi-institutional virtual organizations

6. foster@mcs.anl.gov ARGONNE  CHICAGO Why Grids? (1) eScience l A biochemist exploits 10,000 computers to screen 100,000 compounds in an hour l 1,000 physicists worldwide pool resources for peta-op analyses of petabytes of data l Civil engineers collaborate to design, execute, & analyze shake table experiments l Climate scientists visualize, annotate, & analyze terabyte simulation datasets l An emergency response team couples real time data, weather model, population data

7. foster@mcs.anl.gov ARGONNE  CHICAGO Why Grids? (2) eBusiness l Engineers at a multinational company collaborate on the design of a new product l A multidisciplinary analysis in aerospace couples code and data in four companies l An insurance company mines data from partner hospitals for fraud detection l An application service provider offloads excess load to a compute cycle provider l An enterprise configures internal & external resources to support eBusiness workload

8. foster@mcs.anl.gov ARGONNE  CHICAGO Intelligent Infrastructure: Distributed Servers and Services

9. foster@mcs.anl.gov ARGONNE  CHICAGO Strategic Challenges l Managing the transition from a research area to an industry l Profiting from commercial technologies without compromising eScience needs l Innovating fast enough to meet needs of new application domains l Establishing and extending Grid community l Pursuing the research to address next- generation systems

10. foster@mcs.anl.gov ARGONNE  CHICAGO Our Approach l Define & promote unifying, open view of Grids, integrating commercial technologies –Open Grid Services Architecture l Coherent, high-quality open source code l Pursue realization in close collaboration –For us, Condor Group is first among equals l Reach out to, and engage, industry, emphasizing benefits of collaboration –Identify areas where industry can add value

11. foster@mcs.anl.gov ARGONNE  CHICAGO “Web Services” l Increasingly popular standards-based framework for accessing network applications –W3C standardization; Microsoft, IBM, Sun, others l WSDL: Web Services Description Language –Interface Definition Language for Web services l SOAP: Simple Object Access Protocol –XML-based RPC protocol; common WSDL target l WS-Inspection –Conventions for locating service descriptions l UDDI: Universal Desc., Discovery, & Integration –Directory for Web services

12. foster@mcs.anl.gov ARGONNE  CHICAGO Transient Service Instances l “Web services” address discovery & invocation of persistent services –Interface to persistent state of entire enterprise l In Grids, must also support transient service instances, created/destroyed dynamically –Interfaces to the states of distributed activities –E.g. workflow, video conf., dist. data analysis l Significant implications for how services are managed, named, discovered, and used –In fact, much of our work is concerned with the management of service instances

13. foster@mcs.anl.gov ARGONNE  CHICAGO OGSA Design Principles l Service orientation to virtualize resources –Everything is a service l From Web services –Standard interface definition mechanisms: multiple protocol bindings, local/remote transparency l From Grids –Service semantics, reliability and security models –Lifecycle management, discovery, other services l Multiple “hosting environments” –C, J2EE,.NET, …

14. foster@mcs.anl.gov ARGONNE  CHICAGO OGSA Service Model l System comprises (a typically few) persistent services & (potentially many) transient services –Everything is a service l OGSA defines basic behaviors of services: fundamental semantics, life-cycle, etc. –More than defining WSDL wrappers

15. foster@mcs.anl.gov ARGONNE  CHICAGO Open Grid Services Architecture: Fundamental Structure 1) WSDL conventions and extensions for describing and structuring services –Useful independent of “Grid” computing 2) Standard WSDL interfaces & behaviors for core service activities –portTypes and operations => protocols

16. foster@mcs.anl.gov ARGONNE  CHICAGO Standard Interfaces & Behaviors: Four Interrelated Concepts l Naming and bindings –Every service instance has a unique name, from which can discover supported bindings l Information model –Service data associated with Grid service instances, operations for accessing this info l Lifecycle –Service instances created by factories –Destroyed explicitly or via soft state l Notification –Interfaces for registering interest and delivering notifications

17. foster@mcs.anl.gov ARGONNE  CHICAGO l GridService Required –FindServiceData –Destroy –SetTerminationTime l NotificationSource –SubscribeToNotificationTopic –UnsubscribeToNotificationTopic l NotificationSink –DeliverNotification OGSA Interfaces and Operations Defined to Date l Factory –CreateService l PrimaryKey –FindByPrimaryKey –DestroyByPrimaryKey l Registry –RegisterService –UnregisterService l HandleMap –FindByHandle Authentication, reliability are binding properties Manageability, concurrency, etc., to be defined

18. foster@mcs.anl.gov ARGONNE  CHICAGO GT3: An Open Source OGSA- Compliant Globus Toolkit l GT3 Core –Implements Grid service interfaces & behaviors –Reference impln of evolving standard l GT3 Base Services –Evolution of current Globus Toolkit capabilities l Other Grid services –Many …

19. foster@mcs.anl.gov ARGONNE  CHICAGO User/Application Fabric ( processing, storage, communication ) Grid Condor Globus Toolkit Condor

20. foster@mcs.anl.gov ARGONNE  CHICAGO For More Information l The Globus Project™ –www.globus.org l Grid architecture –www.globus.org/research /papers/anatomy.pdf l Open Grid Services Architecture –www.globus.org/ogsa