2. Globus Toolkit® 4 Ian Foster Argonne National Laboratory University of Chicago Univa Corporation

3. 2 Credits l Globus Toolkit v4 is the work of many talented Globus Alliance members, at u Argonne Natl. Lab & U.Chicago u USC Information Sciences Corporation u National Center for Supercomputing Applns u U. Edinburgh u Swedish PDC u Univa Corporation u Other contributors at other institutions l Supported by DOE, NSF, UK EPSRC, and other sources

4. 3 Don’t take our word for it! Read the UK eScience Evaluation of GT4 www.nesc.ac.uk/technical_papers/UKeS-2005-03.pdf (Reachable from www.globus.org, under “News”)

5. 4 Overview l Background and Globus approach l Globus Toolkit: current capabilities l Future directions l Related tools

6. 5 “… A new age has dawned in scientific and engineering research, pushed by continuing progress in computing, information, and communication technology, and pulled by the expanding complexity, scope and scale of today’s challenges. The capacity of this technology has crossed thresholds that now make possible a comprehensive cyberinfrastructure on which to build new types of scientific and engineering knowledge environments and organizations, and to pursue research in new ways and with increased efficacy…” National Science Foundation Blue Ribbon Advisory Panel, 2003

7. 6 What specific problem is the Globus Toolkit designed to address?

8. 7 Ultimately, the Globus Toolkit is designed to enable the creation and maintenance of Virtual Organizations

9. 8 History In the early 90s, I (Foster) and others (e.g., Carl Kesselman, USC-ISI) enjoyed helping scientists apply distributed computing u Opportunities seemed ripe for the picking u Application of technology always uncovers new and interesting requirements u Science is cool u Big/innovative science is even cooler

10. 9 History (continued) While helping to build/integrate a diverse range of applications, the same problems kept showing up over and over again u Too many different security systems u Too many different scheduling/execution mechanisms u Too many different storage systems u Too many different monitoring/status/event systems

11. 10 What Kinds of Applications? l Computation intensive u Interactive simulation (climate modeling) u Large-scale simulation and analysis (galaxy formation, gravity waves, event simulation) u Engineering (parameter studies, linked models) l Data intensive u Experimental data analysis (e.g., physics) u Image & sensor analysis (astronomy, climate) l Distributed collaboration u Online instrumentation (microscopes, x-ray) Remote visualization (climate studies, biology) u Engineering (large-scale structural testing)

12. 11 Key Common Feature The size and/or complexity of the problem requires that people in several organizations collaborate and share computing resources, data, instruments

13. 12 An Example Problem l The Large Hadron Collider (LHC) l Largest machine ever built by humans! l Located at CERN, Geneva Switzerland l Particle accelerator and collider with a circumference of 16.8 miles l Scheduled to go into production in 2007

14. 13 An Example Problem (continued) l Will generate 10 Petabytes (10 7 Gigabytes) of information per year l This information must be processed and stored somewhere l It is beyond the scope of a single institution to manage this problem

15. 14 R R R R R R R R R R Virtual Organizations Distributed resources and people Linked by networks, crossing admin domains Sharing resources, common goals Dynamic VO-B VO-A R R R R

16. 15 R R R R R R R R R R R R VO-A VO-B Virtual Organizations Distributed resources and people Linked by networks, crossing admin domains Sharing resources, common goals Dynamic Fault tolerant

17. 16 The Globus Approach

18. 17 The Role of the Globus Toolkit l A collection of solutions to problems that come up frequently when building collaborative distributed applications l Heterogeneity u A focus, in particular, on overcoming heterogeneity for application developers l Standards u We capitalize on and encourage use of existing standards (IETF, W3C, OASIS, GGF) u GT also includes reference implementations of new/proposed standards in these organizations

19. 18 Layers in the Grid

20. 19 A Typical eScience Use of Globus: Network for Earthquake Eng. Simulation Links instruments, data, computers, people

21. 20 Without the Globus Toolkit Web Browser Compute Server Data Catalog Data Viewer Tool Certificate authority Chat Tool Credential Repository Web Portal Compute Server Resources implement standard access & management interfaces Collective services aggregate &/or virtualize resources Users work with client applications Application services organize VOs & enable access to other services Database service Database service Database service Simulation Tool Camera Telepresence Monitor Registration Service A B C D E Application Developer 10 Off the Shelf 12 Globus Toolkit 0 Grid Community 0

22. 21 With the Globus Toolkit Web Browser Compute Server Globus MCS/RLS Data Viewer Tool Certificate Authority CHEF Chat Teamlet MyProxy CHEF Compute Server Resources implement standard access & management interfaces Collective services aggregate &/or virtualize resources Users work with client applications Application services organize VOs & enable access to other services Database service Database service Database service Simulation Tool Camera Telepresence Monitor Globus Index Service Globus GRAM Globus DAI Application Developer 2 Off the Shelf 9 Globus Toolkit 4 Grid Community 4

23. 22 The Globus Toolkit: “Standard Plumbing” for the Grid l Not turnkey solutions, but building blocks & tools for application developers & system integrators u Some components (e.g., file transfer) go farther than others (e.g., remote job submission) toward end-user relevance l Easier to reuse than to reinvent u Compatibility with other Grid systems comes for free l Today the majority of the GT public interfaces are usable by application developers and system integrators u Relatively few end-user interfaces u In general, not intended for direct use by end users (scientists, engineers, marketing specialists)

24. 23 The Application-Infrastructure Gap Dynamic and/or Distributed Applications A 1 B 1 9 9 Shared Distributed Infrastructure

25. 24 Provisioning Bridging the Gap: Grid Infrastructure l Service-oriented Grid infrastructure u Provision physical resources to support application workloads Appln Service Users Workflows Composition Invocation l Service-oriented applications u Wrap applications as services u Compose applications into workflows

26. 25 Grid Infrastructure l Distributed management u Of physical resources u Of software services u Of communities and their policies l Unified treatment u Build on Web services framework u Use WS-RF, WS-Notification (or WS-Transfer/Man) to represent/access state u Common management abstractions & interfaces

27. 26 Globus is Open Source Grid Infrastructure l Implement key Web services standards u State, notification, security, … l Software for Grid infrastructure u Service-enable new & existing resources u E.g., GRAM on computer, GridFTP on storage system, custom application services u Uniform abstractions & mechanisms l Tools to build applications that exploit Grid infrastructure u Registries, security, data management, … l Enabler of a rich tool & service ecosystem

28. 27 An eBusiness Use of Globus: SAP Demonstration @ GlobusWorld l 3 Globus-enabled applns: u CRM: Internet Pricing Configurator (IPC) u CRM: Workforce Management (WFM) u SCM: Advanced Planner & Optimizer (APO) l Applications modified to: u Adjust to varying demand & resources u Use Globus to discover & provision resources IPC Dispatcher IPC Server Request: Price Query Delegation of Request Response: Pricelist Depending on: - Time - Discount - Number of Items - … Web Browsers / Batch Processes (typically several thousand requests) IPC Server 1 2 2 3 SAP AG R/3 Internet Pricing & Configurator (IPC)

29. 28 Overview l Background and Globus approach l Globus Toolkit: current capabilities l Future directions l Related tools

30. 29 The Globus Toolkit is a Collection of Components l A set of loosely-coupled components, with: u Services and clients u Libraries u Development tools l GT components are used to build Grid- based applications and services u GT can be viewed as a Grid SDK l GT components can be categorized across two different dimensions u By broad domain area u By protocol support

31. 30 GT Domain Areas l Core runtime u Infrastructure for building new services l Security u Apply uniform policy across distinct systems l Execution management u Provision, deploy, & manage services l Data management u Discover, transfer, & access large data l Monitoring u Discover & monitor dynamic services

32. 31 GT Protocols l Web service protocols u WSDL, SOAP u WS Addressing, WSRF, WSN u WS Security, SAML, XACML u WS-Interoperability profile l Non Web service protocols u Standards-based, such as GridFTP u Custom

33. 32 “Stateless” vs. “Stateful” Services l Without state, how does client: u Determine what happened (success/failure)? u Find out how many files completed? u Receive updates when interesting events arise? u Terminate a request? l Few useful services are truly “stateless”, but WS interfaces alone do not provide built-in support for state Client FileTransfer Service move (A to B) move

34. 33 FileTransferService (without WSRF) l Developer reinvents wheel for each new service u Custom management and identification of state: transferID u Custom operations to inspect state synchronously (whatHappen) and asynchronously (tellMeWhen) u Custom lifetime operation (cancel) Client FileTransfer Service move (A to B) : transferID move state whatHappen tellMeWhen cancel

35. 34 WSRF in a Nutshell l Service l State representation u Resource u Resource Property l State identification u Endpoint Reference l State Interfaces u GetRP, QueryRPs, GetMultipleRPs, SetRP l Lifetime Interfaces u SetTerminationTime u ImmediateDestruction l Notification Interfaces u Subscribe u Notify l ServiceGroups RPs Resource Service GetRP GetMultRPs SetRP QueryRPs Subscribe SetTermTime Destroy EPR

36. 35 FileTransferService (w/ WSRF) l Developer specifies custom method to createResource and leaves the rest to WSRF standards: u State exposed as Resource + Resource Properties and identified by Endpoint Reference (EPR) u State inspected by standard interfaces (GetRP, QueryRPs) u Lifetime management by standard interfaces (Destroy) Client FileTransferService createResource (A to B) : EPR createResource RPs Transfer getRP queryRPs destroy

37. Data MgmtSecurity Common Runtime Execution Mgmt Info Services Non-WS Components Pre-WS Authentication Authorization GridFTP C Common Libraries Globus Toolkit version 2 (GT2) Grid Resource Alloc. Mgmt (GRAM) Monitoring & Discovery (MDS) Web Services Components

38. Data MgmtSecurity Common Runtime Execution Mgmt Info Services Web Services Components Non-WS Components Pre-WS Authentication Authorization GridFTP C Common Libraries WS Authentication Authorization Reliable File Transfer Data Access & Integration Grid Resource Alloc. Mgmt (WS GRAM) MDS3 Java WS Core Community Authorization Replica Location eXtensible IO (XIO) Globus Toolkit version 3 (GT3) Grid Resource Alloc. Mgmt (GRAM) Monitoring & Discovery (MDS)

39. Data MgmtSecurity Common Runtime Execution Mgmt Info Services Web Services Components Non-WS Components Pre-WS Authentication Authorization GridFTP Pre-WS Grid Resource Alloc. & Mgmt Pre-WS Monitoring & Discovery C Common Libraries Authentication Authorization Reliable File Transfer Data Access & Integration Grid Resource Allocation & Management Index Java WS Core Community Authorization Replica Location eXtensible IO (XIO) Credential Mgmt Community Scheduling Framework Delegation Globus Toolkit version 4 (GT4) Data Replication Trigger C WS Core Python WS Core WebMDS Workspace Management Grid Telecontrol Protocol Contrib/ Preview Core Depre- cated www.globus.org

40. 39 Data Mgmt Security Common Runtime Execution Mgmt Info Services GridFTP Authentication Authorization Reliable File Transfer Data Access & Integration Grid Resource Allocation & Management Index Community Authorization Data Replication Community Scheduling Framework Delegation Replica Location Trigger Java Runtime C Runtime Python Runtime WebMDS Workspace Management Grid Telecontrol Protocol Globus Toolkit v4 www.globus.org Credential Mgmt Globus Toolkit: Open Source Grid Infrastructure

41. 40 4.0 is not a typical “.0” release, but the culmination of months of testing 3.0.03.2.0 3.9.5 4.0.03.9.4 3.9.3 3.9.2 3.9.1 3.9.0 3.3.0 3.2.13.0.1 3.0.2 CVS trunk 4.0.1 Stable release branch Development release Stable release

42. 41 Java Services in Apache Axis Plus GT Libraries and Handlers Your Java Service Your Python Service Your Java Service RFT GRAM Delegation Index Trigger Archiver pyGlobus WS Core Your C Service C WS Core RLS Pre-WS MDS CAS Pre-WS GRAM SimpleCAMyProxy OGSA-DAI GTCP GridFTP C Services using GT Libraries and Handlers SERVER CLIENT Interoperable WS-I-compliant SOAP messaging Your Java Client Your C Client Your Python Client Your Java Client Your C Client Your Python Client Your Java Client Your C Client Your Python Client Your Java Client Your C Client Your Python Client X.509 credentials = common authentication Python hosting, GT Libraries GT4 Components

43. 42 Our Goals for GT4 l Usability, reliability, scalability, … u Web service components have quality equal or superior to pre-WS components u Documentation at acceptable quality level l Consistency with latest standards (WS-*, WSRF, WS-N, etc.) and Apache platform u WS-I Basic Profile compliant u WS-I Basic Security Profile compliant l New components, platforms, languages u And links to larger Globus ecosystem

44. 43 Data Mgmt Security Common Runtime Execution Mgmt Info Services GridFTP Authentication Authorization Reliable File Transfer Data Access & Integration Grid Resource Allocation & Management Index Community Authorization Data Replication Community Scheduling Framework Delegation Replica Location Trigger Java Runtime C Runtime Python Runtime WebMDS Workspace Management Grid Telecontrol Protocol Globus Toolkit v4 www.globus.org Credential Mgmt Globus Toolkit: Open Source Grid Infrastructure

45. 44 GT4 Web Services Runtime l Supports both GT (GRAM, RFT, Delegation, etc.) & user-developed services l Redesign to enhance scalability, modularity, performance, usability l Leverages existing WS standards u WS-I Basic Profile: WSDL, SOAP, etc. u WS-Security, WS-Addressing l Adds support for emerging WS standards u WS-Resource Framework, WS-Notification l Java, Python, & C hosting environments u Java is standard Apache

46. 45 GT4 WS Core in a Nutshell RPs Resource Service GetRP GetMultRPs SetRP QueryRPs Subscribe SetTermTime Destroy EPR Implementation of WSRF: Resources, EndpointReferences, ResourceProperties Operation Providers: pre-build implementations of WSRF operations Notification implementation: Topics, TopicSet, Embedded Notification Consumer service Implementations of Resources (ReflectionResource, PersistentReflectionResource) and ResourceProperties (SimpleResourceProperty, ReflectionResourceProperty)

47. 46 GT4 WS Core in a Nutshell RPs Resource Service GetRP GetMultRPs SetRP QueryRPs Subscribe SetTermTime Destroy EPR ResourceHome ResourceHome: The home “owns” the Resource instances in the service SingletonResourceHome: manages single instance of Resource ServiceResourceHome: for services that support a single Resource instance ResourceHomeImpl: manages multiple Resource instances. Supports resources with in- memory state and resources with persistent (on disk) state

48. 47 Service Container GT4 WS Core in a Nutshell RPs Resource Service GetRP GetMultRPs SetRP QueryRPs Subscribe SetTermTime Destroy EPR ResourceHome RPs Resource Service GetRP GetMultRPs SetRP QueryRPs Subscribe SetTermTime Destroy EPR ResourceHome RPs Resource Service GetRP GetMultRPs SetRP QueryRPs Subscribe SetTermTime Destroy EPR ResourceHome Service Container: host multiple services in container; one JVM process …more details: based on AXIS service container, processes SOAP messages, ResourceContext extension.

49. 48 Service Container GT4 WS Core in a Nutshell RPs Resource Service GetRP GetMultRPs SetRP QueryRPs Subscribe SetTermTime Destroy EPR ResourceHome RPs Resource Service GetRP GetMultRPs SetRP QueryRPs Subscribe SetTermTime Destroy EPR ResourceHome RPs Resource Service GetRP GetMultRPs SetRP QueryRPs Subscribe SetTermTime Destroy EPR ResourceHome Secure Communication: Transport, Message, Conversation (Transport demonstrates best performance) PIP PDP Configurable Security Policies: Policy Information Points (PIPs), Policy Decision Points (PDP) -- chained Example authorization PDPs: GridMap, SAML implementations, XACML policies

50. 49 Service Container GT4 WS Core in a Nutshell RPs Resource Service GetRP GetMultRPs SetRP QueryRPs Subscribe SetTermTime Destroy EPR ResourceHome RPs Resource Service GetRP GetMultRPs SetRP QueryRPs Subscribe SetTermTime Destroy EPR ResourceHome RPs Resource Service GetRP GetMultRPs SetRP QueryRPs Subscribe SetTermTime Destroy EPR ResourceHome PIP PDP WorkManagerDB Conn Pool JNDI Directory WorkManager: “thread pool”, site independent “work” manager Apache Database Connection Pool library (JDBC “DataSource” implementation) JNDI Directory: manages internal, shared objects (ResourceHomes, WorkManager, Configuration objects,…)

51. 50 Apache Tomcat Service Container GT4 WS Core in a Nutshell RPs Resource Service GetRP GetMultRPs SetRP QueryRPs Subscribe SetTermTime Destroy EPR ResourceHome RPs Resource Service GetRP GetMultRPs SetRP QueryRPs Subscribe SetTermTime Destroy EPR ResourceHome RPs Resource Service GetRP GetMultRPs SetRP QueryRPs Subscribe SetTermTime Destroy EPR ResourceHome PIP PDP WorkManagerDB Conn Pool JNDI Directory Deploy Service Container “standalone” or within Apache Tomcat

52. 51 Custom Web Services WS-Addressing, WSRF, WS-Notification Custom WSRF Web Services GT4 WSRF Web Services WSDL, SOAP, WS-Security User Applications Registry Administration GT4 Container GT4 Web Services Runtime

53. 52 Stateful Entities Registry Service requestor (e.g., user application) Factory service Create Stateful Entity State Address Resource allocation Register Stateful Entity Discovery Interactions standardized using WSDL and SOAP State inspection Lifetime mgmt Notifications Authentication & Authorization are applied to all requests Modeling State in Web Services

54. 53 WSRF & WS-Notification l Naming and bindings (basis for virtualization) u Every resource can be uniquely referenced, and has one or more associated services for interacting with it l Lifecycle (basis for fault resilient state mgmt) u Resources created by services following factory pattern u Resources destroyed immediately or scheduled l Information model (basis for monitoring, discovery) u Resource properties associated with resources u Operations for querying and setting this info u Asynchronous notification of changes to properties l Service groups (basis for registries, collective svcs) u Group membership rules & membership management l Base Fault type

55. 54 WSRF/WSNs Compared (HPDC 2005) GT4-JavaGT4-CpyGridWareWSRF::LiteWSRF.NET Languages supportedJavaCPythonPerlC#/C++/VBasic, etc. WS-Security password profileYesNoIn progress Yes WS-Security X.509 profileYesIn progressYesIn progressYes WS-SecureConversationYesNoYesNoYes TLS/SSLYes AuthorizationMultiple CalloutNone Persistence of WS-ResourcesYesNot defaultYes Memory FootprintJVM + 10M22 KB12 MB Depends Memory size per WS-Resource Depends on resource state 70B Depends on resource state 0 (file/DB) or 10B (process) Depends on resource state Unmodified hosting environmentYesNoYesYes (Apache)Yes Compliance with WS-I Basic Profile Yes In progressYes Compliance with WS-I Basic Security Profile Yes NoYes LoggingLog4JYes WSE diagnostics WS-ResourceLifetimeYes WS-ResourcePropertiesYes WS-ServiceGroupYes WS-BaseFaultsYes WS-BaseNotificationYesConsumerYesNoYes WS-BrokeredNotificationPartialNo Yes WS-TopicsPartial NoPartial

56. 55 GetRP Test Distributed client and service on same LAN (times in milliseconds) GT4 - JavaGT4 - C pyGridWareWSRF::LiteWSRF.NET No Security GT4 - JavaGT4 - C pyGridWareWSRF::LiteWSRF.NET GT4 - JavaGT4 - C pyGridWareWSRF::LiteWSRF.NET X509 SigningHTTPS 10.05 2.34 25.57 17.1 8.23 181.96 14.8 140.5 81.39 N/A 11.46 2.85 12.91 55.6 149.67

57. 56 GT4 WS Core Performance GT4 JavaGT4 CGT4 PythonWSRF.NET GetRP181.9614.77140.5081.39 SetRP182.0414.99142.2182.48 CreateR188.4614.98132.2696.22 DestroyR182.0315.76136.1286.89 Notify219.51N/A244.93101.57 GT4 JavaGT4 CGT4 PythonWSRF.NET getRP11.462.85149.6712.91 setRP11.472.86150.7912.3 createR18.002.82132.6020.84 destroyR14.922.71149.2116.05 Notify29.269.67169.0745.0 (1) Message-level security (times in milliseconds) (2) Transport-level security (times in milliseconds) “WSRF/WSNs Compared,” HPDC 2005.

58. 57 Data Mgmt Security Common Runtime Execution Mgmt Info Services GridFTP Authentication Authorization Reliable File Transfer Data Access & Integration Grid Resource Allocation & Management Index Community Authorization Data Replication Community Scheduling Framework Delegation Replica Location Trigger Java Runtime C Runtime Python Runtime WebMDS Workspace Management Grid Telecontrol Protocol Globus Toolkit v4 www.globus.org Credential Mgmt Globus Toolkit: Open Source Grid Infrastructure

59. 58 Globus Security l Control access to shared services u Address autonomous management, e.g., different policy in different work-groups l Support multi-user collaborations u Federate through mutually trusted services u Local policy authorities rule l Allow users and application communities to set up dynamic trust domains u Personal/VO collection of resources working together based on trust of user/VO

60. 59 Virtual Organization (VO) Concept l VO for each application or workload l Carve out and configure resources for a particular use and set of users

61. 60 GT4 Security VO Rights Users Rights’ Compute Center Access Services (running on user’s behalf) Rights Local policy on VO identity or attribute authority CAS or VOMS issuing SAML or X.509 ACs SSL/WS-Security with Proxy Certificates Authz Callout: SAML, XACML KCA MyProxy

62. 61 GT4 Security l Public-key-based authentication l Extensible authorization framework based on Web services standards u SAML-based authorization callout l As specified in GGF OGSA-Authz WG u Integrated policy decision engine l XACML policy language, per-operation policies, pluggable l Credential management service u MyProxy (One time password support) l Community Authorization Service l Standalone delegation service

63. 62 GT4’s Use of Security Standards Supported, Supported, Fastest, but slow but insecure so default

64. 63 GT-XACML Integration l eXtensible Access Control Markup Language u OASIS standard, open source implementations l XACML: sophisticated policy language l Globus Toolkit ships with XACML runtime u Included in every client and server built on GT u Turned-on through configuration l … that can be called transparently from runtime and/or explicitly from application … l … and we use the XACML-”model” for our Authz Processing Framework

65. 64 GT Authorization Framework

66. 65 Other Security Services Include … l MyProxy u Simplified credential management u Web portal integration u Single-sign-on support l KCA & kx.509 u Bridging into/out-of Kerberos domains l SimpleCA u Online credential generation l PERMIS u Authorization service callout

67. 66 Data Mgmt Security Common Runtime Execution Mgmt Info Services GridFTP Authentication Authorization Reliable File Transfer Data Access & Integration Grid Resource Allocation & Management Index Community Authorization Data Replication Community Scheduling Framework Delegation Replica Location Trigger Java Runtime C Runtime Python Runtime WebMDS Workspace Management Grid Telecontrol Protocol Globus Toolkit v4 www.globus.org Credential Mgmt Globus Toolkit: Open Source Grid Infrastructure

68. 67 GT4 Data Management l Stage/move large data to/from nodes u GridFTP, Reliable File Transfer (RFT) u Alone, and integrated with GRAM l Locate data of interest u Replica Location Service (RLS) l Replicate data for performance/reliability u Distributed Replication Service (DRS) l Provide access to diverse data sources u File systems, parallel file systems, hierarchical storage: GridFTP u Databases: OGSA DAI

69. 68 GridFTP in GT4 l 100% Globus code u No licensing issues u Stable, extensible l IPv6 Support l XIO for different transports l Striping  multi-Gb/sec wide area transport u 27 Gbit/s on 30 Gbit/s link l Pluggable u Front-end: e.g., future WS control channel u Back-end: e.g., HPSS, cluster file systems u Transfer: e.g., UDP, NetBLT transport Disk-to-disk on TeraGrid

70. 69 Reliable File Transfer: Third Party Transfer RFT Service RFT Client SOAP Messages Notifications (Optional) Data Channel Protocol Interpreter Master DSI Data Channel Slave DSI IPC Receiver IPC Link Master DSI Protocol Interpreter Data Channel IPC Receiver Slave DSI Data Channel IPC Link GridFTP Server l Fire-and-forget transfer l Web services interface l Many files & directories l Integrated failure recovery l Has transferred 900K files

71. 70 Replica Location Service l Identify location of files via logical to physical name map l Distributed indexing of names, fault tolerant update protocols l GT4 version scalable & stable l Managing ~40 million files across ~10 sites Index Local DB Update send (secs) Bloom filter (secs) Bloom filter (bits) 10K<121 M 22410 M 5 M717550 M

72. 71  Cardiff AEI/Golm Birmingham Reliable Wide Area Data Replication Replicating >1 Terabyte/day to 8 sites >30 million replicas so far MTBF = 1 month LIGO Gravitational Wave Observatory www.globus.org/solutions

73. 72 OGSA-DAI l Provide service-based access to structured data resources as part of Globus l Specify a selection of interfaces tailored to various styles of data access—starting with relational and XML

74. 73 MySQL OGSA-DAI service Engine SQLQuery JDBC Data Resources Activities DB2 The OGSA-DAI Framework GZipGridFTPXPath XMLDB XIndice readFile File SWISS PROT XSLT SQL Server Data- bases Application Client Toolkit

75. 74 MySQL OGSA-DAI service Engine SQLQuery JDBC SQL JDBC SQL JDBC SQL JDBC SQL JDBC Multiple SQL GDS SQLQuery Extensibility Example

76. 75 OGSA-DAI: A Framework for Building Applications l Supports data access, insert and update u Relational: MySQL, Oracle, DB2, SQL Server, Postgres u XML: Xindice, eXist u Files – CSV, BinX, EMBL, OMIM, SWISSPROT,… l Supports data delivery u SOAP over HTTP u FTP; GridFTP u E-mail u Inter-service l Supports data transformation u XSLT u ZIP; GZIP l Supports security u X.509 certificate based security

77. 76 OGSA-DAI: Other Features l A framework for building data clients u Client toolkit library for application developers l A framework for developing functionality u Extend existing activities, or implement your own u Mix and match activities to provide functionality you need l Highly extensible u Customise our out-of-the-box product u Provide your own services, client-side support, and data-related functionality

78. 77 Data Mgmt Security Common Runtime Execution Mgmt Info Services GridFTP Authentication Authorization Reliable File Transfer Data Access & Integration Grid Resource Allocation & Management Index Community Authorization Data Replication Community Scheduling Framework Delegation Replica Location Trigger Java Runtime C Runtime Python Runtime WebMDS Workspace Management Grid Telecontrol Protocol Globus Toolkit v4 www.globus.org Credential Mgmt Globus Toolkit: Open Source Grid Infrastructure

79. 78 Execution Management (GRAM) l Common WS interface to schedulers u Unix, Condor, LSF, PBS, SGE, … l More generally: interface for process execution management u Lay down execution environment u Stage data u Monitor & manage lifecycle u Kill it, clean up l A basis for application-driven provisioning

80. 79 GT4 WS GRAM l 2nd-generation WS implementation optimized for performance, flexibility, stability, scalability l Streamlined critical path u Use only what you need l Flexible credential management u Credential cache & delegation service l GridFTP & RFT used for data operations u Data staging & streaming output u Eliminates redundant GASS code

81. 80 GRAM services GT4 Java Container GRAM services Delegation RFT File Transfer request GridFTP Remote storage element(s) Local scheduler User job Compute element GridFTP sudo GRAM adapter FTP control Local job control Delegate FTP data Client Job functions Delegate Service host(s) and compute element(s) GT4 WS GRAM Architecture SEG Job events

82. 81 GRAM services GT4 Java Container GRAM services Delegation RFT File Transfer request GridFTP Remote storage element(s) Local scheduler User job Compute element GridFTP sudo GRAM adapter FTP control Local job control Delegate FTP data Client Job functions Delegate Service host(s) and compute element(s) GT4 WS GRAM Architecture SEG Job events Delegated credential can be: Made available to the application

83. 82 GRAM services GT4 Java Container GRAM services Delegation RFT File Transfer request GridFTP Remote storage element(s) Local scheduler User job Compute element GridFTP sudo GRAM adapter FTP control Local job control Delegate FTP data Client Job functions Delegate Service host(s) and compute element(s) GT4 WS GRAM Architecture SEG Job events Delegated credential can be: Used to authenticate with RFT

84. 83 GRAM services GT4 Java Container GRAM services Delegation RFT File Transfer request GridFTP Remote storage element(s) Local scheduler User job Compute element GridFTP sudo GRAM adapter FTP control Local job control Delegate FTP data Client Job functions Delegate Service host(s) and compute element(s) GT4 WS GRAM Architecture SEG Job events Delegated credential can be: Used to authenticate with GridFTP

85. 84 WS GRAM Performance l Time to submit a basic GRAM job u Pre-WS GRAM: < 1 second u WS GRAM: 2 seconds l Concurrent jobs u Pre-WS GRAM: 300 jobs u WS GRAM: 32,000 jobs l Various studies are underway to test latest software

86. 85 GT4 WS GRAM Performance Number of Client Threads (M) 1248163264128 171529578069 70 2152958797470 64 42958787768695269 85977 726527 69 1677 756427 50 327675686467 647573706665 1288072646371 All numbers are simple jobs/minute, no delegation or staging Sustained Job Load Per Client Thread (N)

87. 86 Workspace Service: The Hosted Activity Policy Client Environment Activity Negotiate access Initiate activity Monitor activity Control activity Interface Resource provider

88. 87 Activities Can Be Nested Policy Client Environment Interface Resource provider Client

89. 88 For Example … Physical machine Procure hardware Hypervisor/OS Deploy hypervisor/OS VM Deploy virtual machine Provisioning, management, and monitoring at all levels JVM Deploy container JVM Deploy service

90. 89 Dynamic Service Deployment Community A Community Z … Community scheduling logic Data distribution Community management Science services... Requirements: Community control Persistence Resource guarantees Non- interference

91. 90 Virtual Machine Costs GRAM job GRAM job in paused VM Job in booted VM

92. 91 Virtual OSG Clusters OSG cluster Xen hypervisors TeraGrid cluster OSG

93. 92 Data Mgmt Security Common Runtime Execution Mgmt Info Services GridFTP Authentication Authorization Reliable File Transfer Data Access & Integration Grid Resource Allocation & Management Index Community Authorization Data Replication Community Scheduling Framework Delegation Replica Location Trigger Java Runtime C Runtime Python Runtime WebMDS Workspace Management Grid Telecontrol Protocol Globus Toolkit v4 www.globus.org Credential Mgmt Globus Toolkit: Open Source Grid Infrastructure

94. 93 Monitoring and Discovery l “Every service should be monitorable and discoverable using common mechanisms” u WSRF/WSN provides those mechanisms l A common aggregator framework for collecting information from services, thus: u MDS-Index: Xpath queries, with caching u MDS-Trigger: perform action on condition u (MDS-Archiver: Xpath on historical data) l Deep integration with Globus containers & services: every GT4 service is discoverable u GRAM, RFT, GridFTP, CAS, …

95. 94 GT4 Container GT4 Monitoring & Discovery GRAMUser MDS- Index GT4 Cont. RFT MDS- Index GT4 Container MDS- Index GridFTP adapter Registration & WSRF/WSN Access Custom protocols for non-WSRF entities Clients (e.g., WebMDS) Automated registration in container WS-ServiceGroup

96. 95 Index Server Performance l As the MDS4 Index grows, query rate and response time both slow, although sublinearly l Response time slows due to increasing data transfer size u Full Index is being returned u Response is re-built for every query l Real question – how much over simple WS-N performance?

97. 96 Information Providers l GT4 information providers collect information from some system and make it accessible as WSRF resource properties l Growing number of information providers u Ganglia, CluMon, Nagios u SGE, LSF, OpenPBS, PBSPro, Torque l Many opportunities to build additional ones u E.g., network monitoring, storage systems, various sensors

98. 97 Java Services in Apache Axis Plus GT Libraries and Handlers Your Java Service Your Python Service Your Java Service RFT GRAM Delegation Index Trigger Archiver pyGlobus WS Core Your C Service C WS Core RLS Pre-WS MDS CAS Pre-WS GRAM SimpleCAMyProxy OGSA-DAI GTCP GridFTP C Services using GT Libraries and Handlers SERVER CLIENT Interoperable WS-I-compliant SOAP messaging Your Java Client Your C Client Your Python Client Your Java Client Your C Client Your Python Client Your Java Client Your C Client Your Python Client Your Java Client Your C Client Your Python Client X.509 credentials = common authentication Python hosting, GT Libraries GT4 Summary

99. GT4 Documentation is Much Improved!

100. 99 Overview l Background and Globus approach l Globus Toolkit: current capabilities l Future directions l Related tools

101. 100 The Globus Commitment to Open Source l Globus was first established as an open source project in 1996 l The Globus Toolkit is open source to: u allow for inspection l for consideration in standardization processes u encourage adoption l in pursuit of ubiquity and interoperability u encourage contributions l harness the expertise of the community l The Globus Toolkit is distributed under the (BSD-style) Apache License version 2

102. 101 The Future: Structure l NSF Community Driven Improvement of Globus Software (CDIGS) project u 5 years of funding for GT enhancement u Regular Globus roadmaps outlining plans l GlobDevhttp://dev.globus.org u Apache-like community development site u Community governance of components u “Globus Toolkit” & other related software u Open for business early 2006 u “Globus Alliance” = “GlobDev committers”

103. 102 GlobDev l The current set of Globus components will be organized into several “Globus Projects” u Projects release products l Each project will have its own group of “Committers” u committers are responsible for governance on matters relating to their products l The “Globus Management Committee” will u provide overall guidance and conflict resolution u approve the creation of new Globus Projects

104. 103 The Future: Content l We now have a solid and extremely powerful Web services base l Next, we will build an expanded open source Grid infrastructure u Virtualization u New services for provisioning, data management, security, VO management u End-user tools for application development u Etc., etc. l And of course responding to user requests for other short-term needs

105. 104 The Future l We now have a solid and extremely powerful Web services base l Next, we will build an expanded open source Grid infrastructure u Virtualization u New services for provisioning, data management, security, VO management u End-user tools for application development u Etc., etc. l And of course responding to user requests for other short-term needs

106. 105 Short-Term Priorities: Security l Improve GSI error reporting & diagnostics l Secure password, one-time password, Kerberos support for initial log on l Trust roots, use of GridLogon l Identity/attribute assertions in GT auth. callouts (e.g., Shib, PERMIS, VOMS, SAML) l Extend CAS admin & policy support l Security logging with management control for audit purposes

107. 106 Short-Term Priorities: Data Management l Space & bandwidth management in GridFTP l Concurrency in globus-url-copy l Priorities in RFT l Data replication service l Enhance policy support in data services l Physical file name creation service l Scalable & distributed metadata manager

108. 107 Short-Term Priorities: Execution Management l Implement GGF JSDL once finalized l Advance reservation support l Policy-driven restart of “persistent” jobs l Improved information collection for jobs l Improved management of job collections l Credential refresh l Development of workspace service l Integration of virtual machines (Xen, VMware) and associated services l Windows port of WS GRAM

109. 108 Short-Term Priorities: Information Services l Many more information sources, including gateways to other systems l Automated configuration of monitoring l Specialized monitoring displays l Performance optimization of registry l Archiver service l Helper tools to streamline integration of new information sources

110. 109 Short-Term Priorities: WS Core l Streamlined container configuration l Remote management interface l Dynamic service deployment l Service isolation: multiple service instances l WS-Notification, subscription performance l Full functionality in C WS Core l Optimized WS-ServiceGroup support l WS-SecureConversation support

111. 110 What to Expect from the Globus Alliance in the Coming Months l Support for users of GT4 u Working to make sure the toolkit meets user needs u Answering questions on the mailing lists u Further improving documentation l Normal evolution of performance, scalability and feature enhancements l Further development of tools and services in support of VOs l Expanding contributions to Globus

112. 111 Overview l Background and Globus approach l Globus Toolkit: current capabilities l Future directions l Related tools

113. 112 The Globus Ecosystem l Globus components address core issues relating to resource access, monitoring, discovery, security, data movement, etc. u GT4 being the latest version l A larger Globus ecosystem of open source and proprietary components provide complementary components u A growing list of components l These components can be combined to produce solutions to Grid problems u We’re building a list of such solutions

114. 113 Many Tools Build on, or Can Contribute to, GT4-Based Grids l Condor-G, DAGman l MPICH-G2 l GRMS l Nimrod-G l Ninf-G l Open Grid Computing Env. l Commodity Grid Toolkit l GriPhyN Virtual Data System l Virtual Data Toolkit l GridXpert Synergy l Platform Globus Toolkit l VOMS l PERMIS l GT4IDE l Sun Grid Engine l PBS scheduler l LSF scheduler l GridBus l TeraGrid CTSS l NEES l IBM Grid Toolbox l …

115. 114 Documenting The Grid Ecosystem The Grid Ecosystem: Software Components for Grid Systems And Applications www.grids-center.org

116. 115 Example Solutions l Portal-based User Reg. System (PURSE) l VO Management Registration Service l Service Monitoring Service l TeraGrid TGCP Tool l Lightweight Data Replicator l GriPhyN Virtual Data System

117. 116 Condor-G l The Condor Project @ U Wisconsin Madison develops software for high-throughput computing on collections of distributed compute resources l Condor-G is an interface to GRAM created by the Condor team that allows users to submit jobs to GRAM servers

118. 117 GridShib l Allows the use of Shibboleth-transported attributes for authorization in GT4 deployments u And, more generally, SAML support l 2 year project started December 1, 2004 l Participants u Von Welch, UIUC/NCSA (PI) u Kate Keahey, UChicago/Argonne (PI) u Frank Siebenlist, Argonne u Tom Barton, UChicago l Beta software released September 16, 2005

119. 118 Handle System l The Handle System from CNRI (http://www.handle.net) is a general- purpose global name service enabling secure name resolution over the internet l The Handle System-GT Integration Project leverages the Handle System for identifier and resolution services through tight integration with GT4’s Web services protocols

120. 119 MPICH-G2 l MPICH-G2, developed at Northern Illinois University and Argonne National Lab, is a grid-enabled implementation of the MPI v1.1 standard l MPICH-G2 is implemented using the pre-WS GRAM component in GT4; integration with GT4 WS GRAM is expected in the near future

121. 120 Nimrod/G l Nimrod is a specialized parametric modeling system from Monash University l Nimrod/G uses a simple declarative parametric modeling language to express parameter sweep experiments. Based on GT4 WS services, Nimrod/G enables the formulation, execution and monitoring of multiple individual parametric experiments

122. 121 Ninf-G4 l Ninf-G4, from AIST, is a reference implementation of the GGF standard GridRPC API l Ninf-G4 is provides higher-level programming APIs for the development and execution of parallel applications on the Grid

123. 122 PERMIS l PERMIS is an EU-funded Privilege Management service that implements Role- Based Access Control l Thanks to the work of the UK Grid Engineering Task Force, services running in a Java WS Core container can use PERMIS via GT4’s SAML authorization callouts

124. 123 SRB l SRB is a package from SDSC providing a uniform interface for connecting to network-based heterogeneous data resources l GT4’s GridFTP includes an interface to SRB data sources, and vice versa

125. 124 Sun Grid Engine l Sun Grid Engine is an open source distributed resource management system from Sun Microsystems l In a collaboration between the London e- Science Centre, Gridwise and MCNC, the Sun Grid Engine has been integrated with GT4

126. 125 Tells Us About Your Grid Tools & Solutions l We list links to related projects on the “Related Software” of the Globus Toolkit web www.globus.org/toolkit/tools/ l “Solutions” are documented on the Globus web www.globus.org/solutions/ l If we’ve got details wrong or you have a GT4-related tool to list on our website, please send mail to info@globus.org

127. 126