1. Globus Toolkit Futures: An Open Grid Services Architecture Ian Foster Carl Kesselman Jeffrey Nick Steven Tuecke Globus Tutorial, Argonne National Laboratory, January 29, 2002

2. June 3, 2015Open Grid Services Architecture2 The Grid World: Current Status l Dozens of major Grid projects in scientific & technical computing/research & education l Considerable consensus on key concepts and technologies –Open source Globus Toolkit™ a de facto standard for major protocols & services –Far from complete or perfect, but out there, evolving rapidly, and large tool/user base l Industrial interest emerging rapidly l Opportunity: convergence of eScience and eBusiness requirements & technologies

3. June 3, 2015Open Grid Services Architecture3 Overview l eBusiness requirements and the need for a services-based model –Jeff Nick, IBM l Open Grid Services Architecture overview –Ian Foster l Grid service specification & demonstration of OGSA prototype –Steven Tuecke

4. Open Grid Services Architecture Overview & Globus Toolkit Evolution Ian Foster Mathematics and Computer Science Division Argonne National Laboratory and Department of Computer Science The University of Chicago

5. June 3, 2015Open Grid Services Architecture5 The Grid Problem Resource sharing & coordinated problem solving in dynamic, multi-institutional virtual organizations

6. June 3, 2015Open Grid Services Architecture6 Why Grids? (1) eScience l A biochemist exploits 10,000 computers to screen 100,000 compounds in an hour l 1,000 physicists worldwide pool resources for peta-op analyses of petabytes of data l Civil engineers collaborate to design, execute, & analyze shake table experiments l Climate scientists visualize, annotate, & analyze terabyte simulation datasets l An emergency response team couples real time data, weather model, population data

7. June 3, 2015Open Grid Services Architecture7 Why Grids? (2) eBusiness l Engineers at a multinational company collaborate on the design of a new product l A multidisciplinary analysis in aerospace couples code and data in four companies l An insurance company mines data from partner hospitals for fraud detection l An application service provider offloads excess load to a compute cycle provider l An enterprise configures internal & external resources to support eBusiness workload

8. June 3, 2015Open Grid Services Architecture8 Intelligent Infrastructure: Distributed Servers and Services

9. June 3, 2015Open Grid Services Architecture9 The Grid World: Current Status l Dozens of major Grid projects in scientific & technical computing/research & education l Considerable consensus on key concepts and technologies –Open source Globus Toolkit™ a de facto standard for major protocols & services –Far from complete or perfect, but out there, evolving rapidly, and large tool/user base l Industrial interest emerging rapidly l Opportunity: convergence of eScience and eBusiness requirements & technologies

10. June 3, 2015Open Grid Services Architecture10 Globus Toolkit: Evaluation (+) l Good technical solutions for key problems, e.g. –Authentication and authorization –Resource discovery and monitoring –Reliable remote service invocation –High-performance remote data access l This & good engineering is enabling progress –Good quality reference implementation, multi- language support, interfaces to many systems, large user base, industrial support –Growing community code base built on tools

11. June 3, 2015Open Grid Services Architecture11 Globus Toolkit: Evaluation (-) l Protocol deficiencies, e.g. –Heterogeneous basis: HTTP, LDAP, FTP –No standard means of invocation, notification, error propagation, authorization, termination, … l Significant missing functionality, e.g. –Databases, sensors, instruments, workflow, … –Virtualization of end systems (hosting envs.) l Little work on total system properties, e.g. –Dependability, end-to-end QoS, … –Reasoning about system properties

12. June 3, 2015Open Grid Services Architecture12 “Web Services” l Increasingly popular standards-based framework for accessing network applications –W3C standardization; Microsoft, IBM, Sun, others l WSDL: Web Services Description Language –Interface Definition Language for Web services l SOAP: Simple Object Access Protocol –XML-based RPC protocol; common WSDL target l WS-Inspection –Conventions for locating service descriptions l UDDI: Universal Desc., Discovery, & Integration –Directory for Web services

13. June 3, 2015Open Grid Services Architecture13 The Need to Support Transient Service Instances l “Web services” address discovery & invocation of persistent services –Interface to persistent state of entire enterprise l In Grids, must also support transient service instances, created/destroyed dynamically –Interfaces to the states of distributed activities –E.g. workflow, video conf., dist. data analysis l Significant implications for how services are managed, named, discovered, and used –In fact, much of our work is concerned with the management of service instances

14. June 3, 2015Open Grid Services Architecture14 Open Grid Services Architecture l Service orientation to virtualize resources l From Web services: –Standard interface definition mechanisms: multiple protocol bindings, multiple implementations, local/remote transparency l Building on Globus Toolkit: –Grid service: semantics for service interactions –Management of transient instances (& state) –Factory, Registry, Discovery, other services –Reliable and secure transport l Multiple hosting targets: J2EE,.NET, “C”, …

15. June 3, 2015Open Grid Services Architecture15 OGSA Service Model l System comprises (a typically few) persistent services & (potentially many) transient services l All services adhere to specified Grid service interfaces and behaviors –Reliable invocation, lifetime management, discovery, authorization, notification, upgradeability, concurrency, manageability l Interfaces for managing Grid service instances –Factory, registry, discovery, lifetime, etc. => Reliable, secure mgmt of distributed state

16. June 3, 2015Open Grid Services Architecture16 The Grid Service l A (potentially transient) Web service with specified interfaces & behaviors, including –Creation (Factory) –Global naming (GSH) & references (GSR) –Lifetime management –Registration & Discovery –Authorization –Notification –Concurrency –Manageability

17. June 3, 2015Open Grid Services Architecture17 Factory l A Grid service with Factory interface can be requested to create a new Grid service instance –Reliable creation (once-and-only-once) l Create operation can be extended to accept Grid-service-specific creation parameters l Returns a Grid Service Handle (GSH) –A globally unique URL –Uniquely identifies the instance for all time –Based on name of a home mapper service

18. June 3, 2015Open Grid Services Architecture18 Mapper l A GSH is a stable name for a Grid service, but does not allow client to actually communicate with the Grid service l A Grid Service Reference (GSR) is a WSDL document that describes how to communicate with the Grid service –Contains protocol binding, network address, … –May expire (I.e. GSR information may change) l The Mapper interface allows a client to map from a GSH to a GSR –http get on GSH also returns a GSR

19. June 3, 2015Open Grid Services Architecture19 Lifetime Management l GS instances created by factory or manually; destroyed explicitly or via soft state –Negotiation of initial lifetime with Factory l SoftStateDestruction interface supports –GetTerminationTime message for inquiry >Notification interface also allows for lifetime notification –SetTerminationTime message for keepalive l Soft state lifetime management avoids –Explicit client teardown of complex state –Resource “leaks” in hosting environments l ExplicitDestruction interface also available

20. June 3, 2015Open Grid Services Architecture20 Discovery l A Grid service instance may maintain a set of service information –XML fragments encapsulated in standard containers l Discovery interface allows clients to query the Grid service instance for this information –Query operation, plus supporting operations >Extensible query language support l See also Notification interfaces –Allows notification of service existence and about service information

21. June 3, 2015Open Grid Services Architecture21 Registry l The Registry interface may be used to discover a set of Grid service instances –Returns a WS-Inspection document containing the GSHs of a set of Grid services –Also returns policy associated with the set –Also available through Discovery interface l The RegistryManagement interface allows for soft-state registration of a Grid service –A set of Grid services can periodically register their GSHs into a registry service, to allow for discovery of services in that set

22. June 3, 2015Open Grid Services Architecture22 Authorization l Protocol binding handles authentication during invocation of Grid service operation –Gives service URI for authenticated subject l Grid service instance should apply authorization policy on all operations –May be site-, service-, instance-, etc., specific l OGSA defines standard interfaces for remote management of access control policy –OperationAuthorizationManagement –SubjectEquivalency

23. June 3, 2015Open Grid Services Architecture23 Notification Interfaces l NotificationSource for client subscription –One or more notification generators >Generates notification message of a specific type >Typed interest statements: E.g., Filters, topics, … >Supports messaging services, 3 rd party filter services, … –Soft state subscription to a generator l NotificationSink for asynchronous delivery of notification messages l A wide variety of uses are possible –E.g. Dynamic discovery/registry services, monitoring, application error notification, …

24. June 3, 2015Open Grid Services Architecture24 Using OGSA to Construct Grid Environments Factory Registry Service Factory H2R Mapper Service... (a) Simple Hosting Environment Factory Registry Service Factory H2R Mapper Service... F R F M SSS F R F M SSS (b) Virtual Hosting Environment E2E Factory E2E Reg E2E H2R Mapper... F 1 R M SSS F 2 R M SSS E2E S (c) Compound Services In each case, Registry handle is effectively the unique name for the virtual organization.

25. June 3, 2015Open Grid Services Architecture25 OGSA and the Globus Toolkit l Technically, OGSA enables –Refactoring of protocols (GRAM, MDS-2, etc.)— while preserving all GT concepts/features! –Integration with hosting environments: simplifying components, distribution, etc. –Greatly expanded standard service set l Pragmatically, we are proceeding as follows –Develop open source OGSA implementation >Globus Toolkit 3.0; supports Globus Toolkit 2.0 APIs –Partnerships for service development –Also expect commercial value-adds

26. June 3, 2015Open Grid Services Architecture26 Globus Toolkit Refactoring l Grid Security Infrastructure (GSI) –Used in Grid service network protocol bindings l Meta Directory Service 2 (MDS-2) –Native part of each Grid service: >Discovery, Registry, RegistryManagement, Notification l Grid Resource Allocation & Mngt (GRAM) –Gatekeeper -> Factory for job mgr instances l GridFTP –Refactor control channel protocol l Other services refactored to used Grid services

27. June 3, 2015Open Grid Services Architecture27 Summary: Evolution of Grid Technologies l Initial exploration (1996-1999; Globus 1.0) –Extensive appln experiments; core protocols l Data Grids (1999-??; Globus 2.0+) –Large-scale data management and analysis l Open Grid Services Architecture (2001-??, Globus 3.0) –Integration w/ Web services, hosting environments, resource virtualization –Databases, higher-level services l Radically scalable systems (2003-??) –Sensors, wireless, ubiquitous computing

28. June 3, 2015Open Grid Services Architecture28 For More Information l The Globus Project™ –www.globus.org l Grid architecture –www.globus.org/research /papers/anatomy.pdf l Open Grid Services Architecture (soon) –www.globus.org/research /papers/ogsa.pdf –www.globus.org/research /papers/gsspec.pdf

29. Grid Service Specification and Demonstration Steven Tuecke Mathematics and Computer Science Division Argonne National Laboratory

30. June 3, 2015Open Grid Services Architecture30 Use of Web Services (1) l A Grid service interface is a WSDL portType l A Grid service definition is a WSDL extension (serviceType) containing: –A set of one or more portTypes supported by the service –portType & serviceType compatibility statements, to support upgradability >For discovery of compatible services when interfaces are upgraded –Implementation version information

31. June 3, 2015Open Grid Services Architecture31 Use of Web Services (2) l A GSR is a WSDL document with extensions: –Extension to service element to reference serviceType –Service element extensions to carry the GSH, and the expiration time of the GSR l A GSH is an URL, with the following properties: –Globally unique for all time –http get on GSH + “.wsdl” returns GSR –Can derive GSH to Mapper from it l Registry returns WS-Inspection documents