1. PAGIS: An Architecture for Programming on the Grid Andrew Wendelborn Distributed & High Performance Computing Group Department of Computer Science, University of Adelaide Adelaide SA 5005, Australia Email: andrew@cs.adelaide.edu.au December 2002

2. Background PAGIS –DHPC propject –Processing of satellite imagery –Started as a compositional process-oriented model for GIS applications –Now an architecture for building grid applications Outline –Base-level API and GUI –Reflective (metalevel) programming for grid applications Framework for defining grid behaviours –Adaptive software: safe reconfiguration –An adaptive resource broker

3. Principles High-level computational model –Programmers shouldn’t need to be grid experts –Reason about behaviour at an abstract level Simple, intuitive interface –Following compositional component-based paradigm Access to system –Programmer abstracts over low-level details … BUT –Wants to tune aspects of system critical to grid application –We provide this via a separate interface

4. Process Networks General, clearly defined semantics Separates meaning from implementation –Different strategies produce equivalent results Compositional Reconfigurable Highly applicable to grid programming Process1 Process3 Process2 Process4

5. Evaluator Client PAGIS overview GUI PN structure

6. Evaluator Scheduler Client

7. Evaluator GRAM - GRIS GIIS Scheduler Grid enabled Globus toolkit Distributed JVMs

8. Tuning and Behaviour We have seen how to build base-level application –How to fine-tune its implementation and behaviour Types of tuning –Replacing code for PN process –Allocation and re-allocation –Monitoring, and other non-application (meta) concerns “Actors” observe and influence behaviour –Human observation and intervention –Other software systems –Change how the computation occurs, not the result of the computation

9. A Reflective Approach Separate concerns into two programs –Functional concerns: baselevel program –Non-functional concerns: metalevel program (meta-behaviour) –Reflects separation in process network model Reify certain aspects of baselevel Metalevel programmer reflects upon selected aspects to control or customize aspects critical to application Software engineering for the grid

10. Method Reification Constructs an explicit representation of a method invocation at the metalevel Provides hooks to control and customize method invocation process We have designed a customization framework, Enigma ObjectAObjectB MetaAMetaB MI

11. Grid Behaviours Enigma gives us a model for describing and implementing interesting grid behaviours Locality –Migrating the computation and its parts Allocation and Adaptation –Computation meta-attributes to drive allocator –Periodically reassess allocation Performance Monitoring –Intercept invocations to monitor progress

12. PAGIS as a Grid Programming Model API provides baselevel interface to system PN provides semantics and safety of adaptation Enigma: hooks to observe / influence behaviour –Software engineering of adaptive applications PN System Implementation Enigma Metalevel Interface Process Network API

13. The System Allocation Manager Channel Service Process Service Distribution Tools Globus Plugin BrokerCOG MDS GRAM Gatekeeper Job Manager Worker JVM fork GRAM data RSL Client Worker LDAP query Client Plugin Java implementation Globus toolkit –Java CoG Coglets for peer-to-peer Java / Globus

14. Current Implementation Building a PN –Java; CoG; Globus toolkit –Operations for simple image processing Inspection of meta-level attributes Changing meta-level attributes –Influences allocation –May lead to migration Underway: integration with DARC

15. DARC provides data abstractions PAGIS provides process abstractions Host A Host B Host C Node DR TCP PAGIS and DARC

16. Client Evaluator GRAM - GRIS DARC GIIS Scheduler DARC provides data sources Powerful abstraction over Globus toolkit

17. Adaptive Resource Broker Project Testbed for different RB strategies Emulator (NISTNet) and simulators NWS (Network Weather Service) –Performance database –Forecasts Detection of anomalous conditions –Generate threshold events –RB adapts its behavior

18. Possible emulation framework

19. Grid Simulator Simple discrete event simulation package –Builds on GridSim and SimGrid –Java, multi-threaded Can specify a full network topology Heterogeneous resources of any kind modelled easily Arbitrary information can be published about resources Allows study of RB techniques on an arbitrary grid –monitored by NWS –threshold events feedback to RB

20. Next Steps Java peer-to-peer environment –Coglets for peer-to-peer Java / Globus DARC integration Adaptive middleware for mobile computing Experimental testbed for Enigma-based behaviours Adaptive RB framework XML intermediate form for PN structure … applications …