The Globus Grid Programming Toolkit: A User-level Tutorial The Globus Project Team ANL and USC/ISI http://www.globus.org

Abstract This tutorial is a practical introduction to programming for high-performance distributed computing systems, or "computational grids," and the capabilities of the Globus grid toolkit. Emerging high-performance networks promise to enable a wide range of emerging application concepts such as remote computing, distributed supercomputing, tele-immersion, smart instruments, and data mining. However, the development and use of such applications is in practice very difficult and time consuming, because of the need to deal with complex and highly heterogeneous systems. The Globus grid programming toolkit is designed to help application developers and tool builders overcome these obstacles to the construction of "grid-enabled” scientific and engineering applications. It does this by providing a set of standard services for authentication, resource location, resource allocation, configuration, communication, file access, fault detection, and executable management. These services can be incorporated into applications and/or programming tools in a "mix-and-match" fashion to provide access to needed capabilities. Our goal in this tutorial is both to introduce the capabilities of the Globus toolkit and to show attendees how Globus services can be applied in specific applications. Hence, the tutorial covers a mixture of grid programming principles and detailed case studies of real applications.

Tutorial Goals Provide an introduction Enable attendees To the structure of the Globus computational grid To the capabilities of the Globus toolkit To pragmatic issues associated with using the toolkit Enable attendees To start building & using Globus applications To utilize Globus services

Overview Introduction to computational grids High-level overview of the Globus toolkit Four components: Security and remote process creation Running programs across multiple resources Information services Dynamic configuration and resource management Case studies Other Globus services, and future directions Globus installation & administration

Why “The Grid”? New applications based on high-speed coupling of people, computers, databases, instruments, etc. Computer-enhanced instruments Collaborative engineering Browsing of remote datasets Use of remote software Data-intensive computing Very large-scale simulation Large-scale parameter studies

E.g.: Computer-Enhanced Instruments for Microtomography Coupling with supercomputers Interactive use of beamline Collaboration on results Parameter studies for experiment planning Coupling with mass store systems 50 Mb/s -> 5 Gb/s -> 100 Gb/s APS beamline @ Argonne “100 Gflop/sec, 50 Mb/sec, 5 minutes; rendering, 10 GB storage” 5 Mb/s -> 1 Gb/s -> 10 Gb/s Los Angeles Chicago

E.g.: Tele-immersion “5 Gflop/sec, flowspecs, design db” Multiple access modalities Multiple flows Control Text Video Audio Database Simulation Tracking Haptics Rendering Leigh et al., UofI, Electronic Visualization Lab.

SF-Express: Distributed Interactive Simulation Caltech Exemplar NCSA Origin Issues: Resource discovery, scheduling Configuration Multiple comm methods Message passing (MPI) Scalability Fault tolerance Maui SP Argonne SP “200 GB memory, 100 BIPs” P. Messina et al., Caltech

The Grid “Dependable, consistent, pervasive access to [high-end] resources” Dependable: Can provide performance and functionality guarantees Consistent: Uniform interfaces to a wide variety of resources Pervasive: Ability to “plug in” from anywhere

Evolution of a Concept Metacomputing: late 80s Focus on distributed computation Gigabit testbeds: early 90s Research, primarily on networking I-WAY: 1995 Demonstration of application feasibility PACIs (National Technology Grid): 1998 NASA Information Power Grid: 1999 ASCI DISCOM: 1999; SSI: 2000?

National and International Grid Testbeds I-WAY The Alliance National Technology Grid National and International Grid Testbeds NASA’s Information Power Grid

Technical Challenges Complex application structures, combining aspects of parallel, multimedia, distributed, collaborative computing Dynamic varying resource characteristics, in time and space Need for high & guaranteed “end-to-end” performance, despite heterogeneity and lack of global control Interdomain issues of security, policy, payment

Issues Authenticate once Specify simulation (code, resources, etc.) Locate resources Negotiate authorization, acceptable use, etc. Acquire resources Initiate computation Steer computation Access remote datasets Collaborate on results Account for usage Domain 1 Domain 2

Architectural Approaches Distributed systems: DCE, CORBA, Jini, etc. Rich functionality eases app. development Complexity hinders deployment especially in absence of global control Performance difficulties Internet Protocol, Web tools Simple protocols facilitate deployment Missing functionality hinders app. development

Standards & Commodity Tech Where appropriate, exploit standards and commodity technology in core infrastructure LDAP, SSL, X.509, GSS-API, GAA-API, http, ftp, XML, etc. Provides leverage Interface with other common standards CORBA, Java/Jini, DCOM, Web, etc While our core infrastructure may not be built on one of these distributed architectures, we must cleanly interface with them

The Globus Project Basic research in grid-related technologies Resource management, QoS, networking, storage, security, adaptation, policy, etc. Development of Globus toolkit Core services for grid-enabled tools & applns Construction of large grid testbed: GUSTO Largest grid testbed in terms of sites & apps Application experiments Tele-immersion, distributed computing, etc.

Globus Approach A toolkit and collection of services addressing key technical problems Bag of services model Not a vertically integrated solution Inter-domain issues, rather than clustering Integration of intra-domain solutions Distinguish between local and global services “IP hourglass” model

Technical Focus & Approach Information-rich environment Basis for configuration and adaptation Enable incremental development of grid-enabled tools and applications Support many programming models, tools, applications Deploy toolkit on national-scale testbed to allow large-scale applications Evolve in response to user requirements

Globus Approach Focus on architecture issues Design principles Propose set of core services as basic infrastructure Use to construct high-level, domain-specific solutions Design principles Keep participation cost low Enable local control Support for adaptation A p p l i c a t i o n s Diverse global services Core Globus services Local OS

Layered Architecture Applications High-level Services and Tools GlobusView Testbed Status DUROC MPI MPI-IO CC++ Nimrod/G globusrun Core Services Nexus GRAM Metacomputing Directory Service Globus Security Interface Heartbeat Monitor Gloperf GASS Local Services LSF Condor MPI NQE Easy TCP Solaris Irix AIX UDP

Core Globus Services Communication infrastructure (Nexus, IO) Information services (MDS) Network performance monitoring (Gloperf) Process monitoring (HBM) Remote file and executable management (GASS and GEM) Resource management (GRAM) Security (GSI)

Sample of High-Level Services I Communication & I/O libraries MPICH, PAWS, RIO (MPI-IO), PPFS, MOL Parallel languages CC++, HPC++ Collaborative environments CAVERNsoft, ManyWorlds Others MetaNEOS, NetSolve, LSA, AutoPilot, WebFlow

Sample High-Level Services II Resource brokers and co-allocators DUROC: co-allocation of multiple systems Nimrod: high-throughput computing Graphical system status display elements GlobusView MDS Browsers Health & Status Monitors (HBM) Network Monitors (Gloperf)

“GUSTO” Globus Ubiquitous Supercomputing Testbed Organization A collection of organizations committed to creating a persistent computational grid infrastructure As of November 1998, 70 organizations in 3 continents and 8 countries

16 sites, 330 computers, 3600 nodes, 2 Teraflop/s, 10 application partners

GUSTO Testbed During SC’97

GUSTO Computational Grid Testbed: November 1998

Where We Are (November 1998) New results in security, resource management, tools, fault detection, etc. Globus v1.0 completed All core services complete, relatively robust, and documented Available on most Unix platforms Many tool projects are leveraging this considerable investment in infrastructure Interesting applications are emerging, although mostly still in “demo” mode

Where We Are (June 1999) New results in QoS, security, resource management, data management, tools, etc. Globus v1.1 nearing completion Available on most Unix platforms and Win32 Many tool projects are leveraging this considerable investment in infrastructure Documentation and deployment underway at NCSA and NASA IPG Always looking for interesting applications

Changes from 1.0 to 1.1 Tutorial changes for 1.1 are denoted by Name changes from Globus to Grid Security and Information Service adopted as core Grid infrastructure by several organizations Globus Security Infrastructure -> Grid Security Infrastructure Metacomputing Directory Service -> Grid Information Service Affects naming of APIs and tools Numerous small API fixes, additions, changes Cleanup of programs/tools A few new modules (I/O, error objects)

Example Application Projects Computed microtomography (ANL, ISI) Real-time, collaborative analysis of data from X-Ray source (and electron microscope) Hydrology (ISI, UMD, UT; also NCSA, Wisc.) Interactive modeling and data analysis Collaborative engineering (“tele-immersion”) CAVERNsoft @ EVL, Metro @ ANL X-Ray crystallography (ANL, SUNY) High-throughput computing for Shake ‘n Bake

Example Application Expts (contd) Distributed interactive simulation (CIT, ISI) Record-setting SF-Express simulation Remote visualization and steering for astrophysics Including trans-Atlantic experiments Data-intensive computing experiments (with LBNL and SLAC: “Clipper” project)

For More Information on Globus http://www.globus.org Papers on all components Tutorial and documents Software Application descriptions

The Grid: Blueprint for a New Computing Infrastructure I. Foster, C The Grid: Blueprint for a New Computing Infrastructure I. Foster, C. Kesselman (Eds), Morgan Kaufmann, 1999 Available July 1998; ISBN 1-55860-475-8 22 chapters by expert authors including Andrew Chien, Jack Dongarra, Tom DeFanti, Andrew Grimshaw, Roch Guerin, Ken Kennedy, Paul Messina, Cliff Neuman, Jon Postel, Larry Smarr, Rick Stevens, and many others “A source book for the history of the future” -- Vint Cerf http://www.mkp.com/grids

Tutorial Approach Four sections, each illustrates a basic Globus technique Laboratory material is available to allow practice with the use of each technique See http://www.globus.org/tutorial