1. Globus Overview CS-780-3 Lecture Notes In courtesy of Andy Kowalski

2. Overview What Defines a Grid Globus GT1, GT2 (pre OGSA) GT3 (post OGSA) Issues

3. What Defines a Grid? “We will probably see the spread of ‘computer utilities, which, like present electric and telephone utilities, will service individual homes and offices across the country” -Len Kleinrock. 1969 [1]

4. What Defines a Grid? “A computational grid is a hardware and software infrastructure that provides dependable, consistent, pervasive, and inexpensive access to high-end computational capabilities.” - Ian Foster and Carl Kesselman, 1998 [2]

5. Ian Foster’s Three Point Checklist (2002) A Grid is a system that: 1. “coordinates resources that are not subject to centralized control” [2] 2. “using standard, open, general- purpose protocols and interfaces” [2] 3. “to deliver nontrivial quantities of service” [2]

6. Globus Project Multi-institutional research effort on the technology required to build a grid or grids  Mathematics and Computer Science Division at Argonne National Laboratory  Information Sciences Institution at the University of Southern California  University of Chicago  University of Edinburgh  Center for Parallel Computers Principle Investigators Ian Foster - Argonne National Laboratory and University of Chicago Carl Kesselman - University of Southern California

7. Globus Project Seek to understand application requirements for a usable grid and … Conduct research and development to create the technologies required to build a grid Globus toolkit

8. Globus Toolkit Bag of services Developers can use these to build grid applications Makes use of existing standards when possible Interfaces and implementations Distinguishes between local and global services Defines interfaces to manage heterogeneity Does not hide heterogeneity Information service to deal with constant changes Utilization, failures, new resources, etc.

9. Hourglass Principle A layered architecture with and hourglass shape Grid services should provide a simple well-defined interface between applications and local services

10. GRAM Globus Resource Allocation Manager Provides a single interface for requesting and using remote system resources for executing jobs Each GRAM is responsible for the resource management systems local to it LSF (Load Sharing Facility), Condor, PBS Provides a standard interface to these different systems Applications can access these different systems using a standard API Within GRAM resource requests are described by the Resource Specification Language (RSL)

11. GRAM Application Broker Information Service LSF Co-allocator GRAM PBS GRAM RSL Queries & Info Ground RSL Single Ground RSL - Ground RSL is an expanded description of the application’s request. It may represent more than on job. - Single Ground RSL is a description of a singe job created from the Ground RSL. There may be more than one created from the original Ground RSL.

12. RSL Resource Specification Language Provides a common language to describe resources Uses attribute, value pairs A simple example & (executable = a.out (* <-- that is an unquoted literal *)) (directory = /home/nobody ) (arguments = arg1 "arg 2") (count = 1)

13. Nexus Low-level communications API that supports other communication libraries Can select alternative low-level protocols based on network topology Message passing, IP, shared memory Can introduce encryption based on source and destination of a message Intended for use by compiler writers and library developers CC++, HPC++, and MPICH-G

14. MDS Metacomputing Directory Service (GT1) Monitoring and Discovery Service (GT2&3) Provides information about the systems in the grid CPU type, memory, OS, network, etc. Queue information, utilization, etc Basically a local LDAP server MDS ties all the LDAP servers together so everyone has access to the same information Has an API for discovering, publishing, and accessing information

15. MDS Architechture Grid Index Information Service (GIIS) Provides an aggregate directory of lower level data Grid Resource Information Service (GRIS) Runs on a resource and acts like a content gateway to the resource Information Provider (IP) Interface to data collection services

16. GSI Globus Security Infrastructure Deals only with authentication Users and services Uses x.509 certificates Also used username and password (rlogin) Uses Secure Socket Layer (SSL) for the connection and authentication No longer true with GT3 All Globus services use GSI to authenticate users

17. GSI Use proxy certificates for non-interactive work Jobs executing on remote systems Proxy certificates have a limited life time

18. CAS Community Authorization Service In the CAS model, resource providers grant access to blocks of resources to a community as a whole, and the community uses a CAS server to perform fine-grained access control on those resources.  Resource providers grant course-grained access to communities.  Communities run CAS servers, which keep track of fine- grained access control information and grant restricted proxies to community members.  The result is that a CAS user gets the intersection of the rights granted by resource provider to the community and the rights granted by the community to that user. Slide Taken From [6]

19. A Typical CAS Request 2.CAS reply, including restricted proxy cred: CAS Server What rights does the community grant to this user? User 1.CAS request, authenticated with Resource Server Do the proxy restrictions authorize this request? 3. Resource request, authenticated with CAS proxy 4. Resource reply CAS-maintained community policy database User credential Community subject name Is this request authorized for the community? Local policy information Policy restrictions Slide Taken From [6]

20. HBM Heartbeat Monitor API to register with the service and collect data Service expects updates from clients that have registered If no update is received, attempts to determine if the service itself is down or if there are other factors

21. GASS Global Access to Secondary Storage Allows programs to use the standard C I/O library to read and write files on remote computers Reads File is copied locally and then read Writes File is written locally and then copied to their destination Appends Communicated to the remote file as soon as it is written

22. GEM Globus Executable Management Support identification, location, and creation of executables in heterogeneous environments Pre-stages executables to multiple remote sites Removes them after execution

23. GridFTP Secure data transfer protocol GSI authentication Optimized for high-bandwidh wide-area networks Parallel streams - multiple data channels Partial file transfer Third-party transfers Server to server

24. Globus Toolkit 3 Based on the Open Grid Services Architecture (OGSA) Attempts to leverage industry standards Makes use of web services Preserves backward compatibility with GT2 Future releases will begin eliminating compatibility Client interfaces to services are now WSDL/OGSI based

25. Issues Protocols and interfaces Where the GT1 and GT2 implementation of a grid service was inadequate for certain grids, others would developed competing services using different interfaces and protocols  Defeated the goal of compatibility OGSA/OGSI and web services are hoped to resolve this Don’t need the whole thing Want to use GSI, need to install all of Globus COGs helped eliminate this burden  Use only what you need

26. Issues Globus originally seemed focused on compute grids Other groups were building data grids data grids represent a larger customer base GridFTP was over extended Was once proposed that it be used for more than file transfers  Message passing VO Management

27. A few of the Grid Projects with strong HEP collaboration US projects European projects Many national, regional Grid projects -- GridPP(UK), INFN-grid(I), NorduGrid, Dutch Grid, … Slide Taken From [8]

28. The Large Hadron Collider Project 4 detectors CMS ATLAS LHCb Storage – Raw recording rate 0.1 – 1 GBytes/sec Accumulating at 5-8 PetaBytes/year 10 PetaBytes of disk Processing – 200,000 of today’s fastest PCs Slide Taken From [8]

29. 2.5-10Gbs 1-10Gbs Multi-Tiered View of LHC Computing Slide Taken From [8]

30. Jlab Work Storage Resource Manager (SRM) Collaboration with LBL and FNAL Standard interface to a mass storage system  Jasmine, Enstore, HPSS Uses  GSI for authentication  Web Services (before OGSA)

31. Jlab Work Replica Catalog Web Services (before OGSA) Does not use GSI Mysql database used to store the map  GFN -> SURL1 … SURLN Only one server/service  May deploy mirrors of the database and proxies User Job Description Language (uJDL) Collaboration with BNL High level description of a physics job

32. Jlab Work Job Scheduling Project Accepts a job description from a user Creates on or more jobs for execution If required, moves all necessary data to the site where execution will take place Moves output files to specified site, location Will use the uJDL but need to also define a System Job Description Language  Lower level description of a compute job

33. Job Scheduler Design

34. Current Replica Catalog Replica Catalog GSI Site A Site CSite D Site B VO Manager GSI 1 2 3 5 6 87 4 SRM GSI GridFTP Server GSI SRM GSI GridFTP Server GSI SRM GSI GridFTP Server GSI

35. Proposed Replica Catalog Project Replica Catalog GSI Replica Catalog GSI Replica Catalog GSI Replica Catalog GSI P2P Overlay Network Site A Site CSite D Site B VO Manager GSI 1 2 4 3 5 7 8 109 6 SRM GSI GridFTP Server GSI SRM GSI GridFTP Server GSI SRM GSI GridFTP Server GSI

36. Another Replica Catalog Problem Given Multiple VOs want to share data files Each VO represents a country or geographic area Each VO will maintain their own Replica Catalog Problem All VOs that want to participate need seamless access to all Replica Catalogs Individual VOs need to be able to come and go as they please

37. Another Replica Catalog Problem Replica Catalog GSI P2P Overlay Network Replica Catalog GSI Replica Catalog GSI Replica Catalog GSI USA Canada EuropeJapan Replica Catalog GSI Replica Catalog GSI Replica Catalog GSI Replica Catalog GSI

38. References [1] Ian Foster. What is the Grid? A Three Point Checklist. July 20, 2002. [2] Ian Foster and Carl Kesselman, editiors. “The Grid: Blueprint for a New Computing Infrastructure.” 1998. [3] Ian Foster and Carl Kesselman. The Globus Project: A Status Report. Proceedings of IPPS/SPDP’98 Heterogeneous Computing Workshop. 1998. [4] Ian Foster and Carl Kesselman. Globus: Ametacomputing Infrastructure Toolkit. International Journal of Supercomputer Applications. 1997 [5] Globus Project. Globus Toolkit 2.2 MDS Technology Brief. January 30, 2003. [6] Globus Project. Community Authorization Service (CAS) Overview. December 18, 2001. [7] Ian Foster. Open Grid Services Architecture. Plenary Talk at CHEP 2003. March 26, 2003. [8] Ian Bird. Deploying the LHC Computing Grid. Plenary Talk at CHEP 2003. March 26, 2003.