The Globus Project: A Status Report Ian Foster Carl Kesselman

2 Why “The Grid”? l New applications based on high-speed coupling of people, computers, databases, instruments, etc. u Computer-enhanced instruments u Collaborative engineering u Browsing of remote datasets u Use of remote software u Data-intensive computing u Very large-scale simulation u Large-scale parameter studies

3 SF-Express: Distributed Interactive Simulation P. Messina et al., Caltech l Issues: u Resource discovery, scheduling u Configuration u Multiple comm methods u Message passing (MPI) u Scalability u Fault tolerance NCSA Origin Caltech Exemplar Argonne SP Maui SP “ 200 GB memory, 100 BIPs ”

4 The Grid “Dependable, consistent, pervasive access to [high-end] resources” l Dependable: Can provide performance and functionality guarantees l Consistent: Uniform interfaces to a wide variety of resources l Pervasive: Ability to “plug in” from anywhere

5 Technical Challenges l Complex application structures, combining aspects of parallel, multimedia, distributed, collaborative computing l Dynamic varying resource characteristics, in time and space l Need for high & guaranteed “end-to-end” performance, despite heterogeneity and lack of global control l Interdomain issues of security, policy, payment

6 The Globus Project l Basic research in grid-related technologies u Resource management, QoS, networking, storage, security, adaptation, policy, etc. l Development of Globus toolkit u Core services for grid-enabled tools & applns l Construction of large grid testbed: GUSTO u Largest grid testbed in terms of sites & apps l Application experiments u Tele-immersion, distributed computing, etc.

7 Globus Approach l A toolkit and collection of services addressing key technical problems u Bag of services model u Not a vertically integrated solution l Distinguish between local and global services u “IP hourglass” model

8 Globus Approach l Focus on architecture issues u Propose set of core services as basic infrastructure u Use to construct high-level, domain-specific solutions l Design principles u Keep participation cost low u Enable local control u Support for adaptation Diverse global services Core Globus services Local OS A p p l i c a t i o n s

9 Layered Architecture Applications Core Services Metacomputing Directory Service GRAM Globus Security Interface Heartbeat Monitor Nexus Gloperf Local Services LSF CondorMPI NQEEasy TCP SolarisIrixAIX UDP High-level Services and Tools DUROCglobusrunMPINimrod/GMPI-IOCC++ GlobusViewTestbed Status GASS

10 Core Globus Services l Communication infrastructure (Nexus, IO) l Information services (MDS) l Network performance monitoring (Gloperf) l Process monitoring (HBM) l Remote file and executable management (GASS and GEM) l Resource management (GRAM) l Security (GSI)

11 Sample of High-Level Services l Communication & I/O libraries u MPICH, PAWS, RIO (MPI-IO), PPFS, MOL l Parallel languages u CC++, HPC++ l Collaborative environments u CAVERNsoft, ManyWorlds l Others u MetaNEOS, NetSolve, LSA, AutoPilot, WebFlow

12 GUSTO Computational Grid Testbed: November 1998

13 Example Application Projects l Real-time, collaborative analysis of data from X- Ray source (and electron microscope) l Interactive modeling and data analysis l Collaborative engineering (“tele-immersion”) u EVL, ANL l Distributed interactive simulation u Record-setting SF-Express simulation l Remote visualization and steering for astrophysics u Including trans-Atlantic experiments