Ames Research CenterDivision 1 Information Power Grid (IPG) Overview Anthony Lisotta Computer Sciences Corporation NASA Ames May 2, 2003

Ames Research CenterDivision 2 Outline What are Grids? Current State of Information Power Grid (IPG) Future Directions Wrap-up

Ames Research CenterDivision 3 Grids A Grid is a system that*: 1.coordinates resources that are not subject to centralized control … 2.… using standard, open, general-purpose protocols and interfaces … 3.… to deliver nontrivial qualities of service. Grid software is “middleware” Resources Networks This is a Grid Enabled Infrastructure * - Ian Foster, July 2002

Ames Research CenterDivision 4 Normal Grid Characteristics An underlying security infrastructure such as the Grid Security Infrastructure (GSI), which is based on public key technology –Protection for at least authentication information as it flows from resource to resource Readily accessible information about the resources on the Grid via a single mechanism, the Grid Information Service (GIS) Single sign-on A seamless processing environment An infrastructure that is scalable to a large number of resources The ability for the grid to cross administrative boundaries

Ames Research CenterDivision 5 The Grid Today Common Middleware -Abstracts independent, hardware, software, user ids, into a service layer with defined APIs -Provides comprehensive security, -Allows for site autonomy -Provides a common infrastructure based on middleware Grid Middleware User Application Site A Site B Network Infrastructure 1 Request info from the grid 1 2 Get response 2 3 Make selection and submit job 3 The underlying infrastructure is abstracted into defined APIs thereby simplifying developer and user access to resources, however, this layer is not intelligent.

Ames Research CenterDivision 6 Outline What are Grids? Current State of Information Power Grid (IPG) Future Directions Wrap-up

Ames Research CenterDivision 7 IPG Vision To make the practice of large-scale science and engineering, as well as other widely distributed, data intensive NASA activities, much more effective than it is today. Grid technology is the foundation to making this vision a success

Ames Research CenterDivision 8 IPG Locations Ames Research Center Jet Propulsion Labs Glenn Research Center Langley Research Center Goddard Space Flight Center

Ames Research CenterDivision 9 LSF Globus GRAM Grid FTP MDS GRISPBS Globus GRAM Grid FTP MDS GRIS MDS GIIS GRAM Job Job Manager Job:  Files to pre-stage  Application to execute  Files to post-stage IPG Job Manager Client IPG Job Manager Globus GRAM Client 3 rd party copy Grid FTP Client Current IPG Services Launch Pad Web Portal IPG Resource Broker MDS Client IPG Resource Broker Client Input: System requirements How to use systems Output: Suggested Job Manager Jobs resource information

Ames Research CenterDivision 10 Outline What are Grids? Current State of Information Power Grid (IPG) Future Directions Wrap-up

Ames Research CenterDivision 11 Future Directions Focus on Managing Complex, Distributed Data (MCDD) –Intelligently manage data in a grid IPG Advanced Services –easier to use –deliver value added services –easier to maintain

Ames Research CenterDivision 12 MCDD Goal: Intelligently manage data in a grid NASA data is inherently distributed e.g., various Earth science archives. Important focus of IPG Access to files –Initial use of grid-enabled Storage Resource Broker –Data staging and replica management building on grid community research –Need grid support for file metadata NASA data can be on –Disk-resident data pools –Tertiary storage data archives Will build on considerable data grid work from the international grid community

Ames Research CenterDivision 13 MCDD - Objectives Identify key data services abstractions that will address a wide range of large-scale, federated, distributed NASA data problems –This potentially has considerable utility in many NASA areas as it essentially seeks to abstract, regularize, and componentize the general operations that are performed on scientific data. Analyze these abstractions in the context of services oriented Grids and determine an implementation and prototyping strategy –Identify existing work in this area and design an implementations approach that maximizes leverage from current work Demonstrate the prototype services within the context of some existing NASA applications

Ames Research CenterDivision 14 IPG Advanced Services Goal: Location-independent computing Intelligent resource selection –User-specified preferences and requirements –Resource characteristics, access, allocations, cost, load, performance predictions Create system-dependent job from system-independent job –Given a system (or systems) to use –Pick directories, set paths and environment, specify which executables and libraries to move, specify which data files to move Reliable execution of system-dependent job –Application execution & file management Dynamic access and accounting –Run applications without a permanent local account –Charging resource use to remote accounting systems –Resource pricing Workflow management –Specify and execute workflows

Ames Research CenterDivision 15 Concept Architecture Job Manager Job User Program Portability Manager Portability Manager Client Portability Manager Job Resource Selector Client Resource Broker Client Resource Broker Resource Selector resource requirements and preferences Job Manager Client Resource Broker Job resource sets Dynamic Access Client Allocation Manager Performance Prediction Clients GIS Client Data Grid Client PBS Globus GRAMGrid FTP GRAM Job IPG Job Manager Globus GRAM Client 3 rd party copy Grid FTP Client Testing and Monitoring resource info allocationsresource and service status Resource Pricing prices Access Policies Allocation Client Access Client deduct from allocation access allowed map user to userid access allowed Higher-level Services In Development Now Transitioned Services Higher-level Services Version 1 in use

Ames Research CenterDivision 16 Outline What are Grids? Current State of Information Power Grid (IPG) Future Directions Wrap-up

Ames Research CenterDivision 17 Summary Stability level Testbed Learning Grids Organizing Players Persistent Grid Informal Organization Lots of Teamwork Shift primary work to increase usability Globus stabilized Attempt inter-grid processing Begin development of initial services Transition support of IPG to operations Testbed in parallel using GT3? Formalize NASA grid organizations Continue to make the Grid easier to use Research complex distributed data management 2004 Testbed Operations Transition Period