Nimrod/G GRID Resource Broker and Computational Economy David Abramson, Rajkumar Buyya, Jon Giddy School of Computer Science and Software Engineering Monash University, Melbourne, Australia {davida, rajkumar,

2100 Desktop (Single Processor?) SMPs or SuperCom puters Local Cluster Global Cluster/Grid PERFORMANCEPERFORMANCE Computing Platforms Evolution Breaking Administrative Barriers Inter Planet Cluster/Grid ?? Individual Group Department Campus State National Globe Inter Planet Universe Administrative Barriers Enterprise Cluster/Grid ?

Domain 2 Domain 1 Grid Resource Management: Challenging Issues Ack.: globus.. Authentication (once) Specify simulation (code, resources, etc.) Discover resources Negotiate authorization, acceptable use, Cost, etc. Acquire resources Schedule Jobs Initiate computation Steer computation Access remote data-sets Collaborate on results Account for usage

Grid Components Grid Fabric Networked Resources across Organisations Computers Clusters Data Sources Scientific Instruments Storage Systems Local Resource Managers Operating Systems Queuing Systems TCP/IP & UDP … Libraries & App Kernels … Distributed Resources Coupling Services Comm. Sign on & SecurityInformation … QoS Process Data Access Development Environments and Tools Languages Libraries Debuggers … Web tools Resource BrokersMonitoring Applications and Portals Prob. Solving Env. Scientific … Collaboration Engineering Web enabled Apps Grid Apps. Grid Middleware Grid Tools

Grid Node N Grid Node 2 Computational Market Model for Grid Resource Management Grid User Application Grid Resource Broker Grid Resource/Control Domains Grid Explorer Schedule Advisor Trade Manager Job Control Agent Deployment Agent Trade Server Resource Allocation Resource Reservation R1R1 Other services Grid Information Server(s) R2R2 RmRm … Charging Alg. Accounting Grid Node1 … Trading Grid Middleware … Info ? … Jobs Health Monitor

 A global scheduler for managing and steering task farming (parametric simulation) applications on computational grid based on deadline and computational economy.  Key Features – A single window to manage & control experiment – Resource Discovery – Trade for Resources – Scheduling – Steering & data management  It allows to study the behaviour of some of the output variables against a range of different input scenarios. What is Nimrod/G ?

Nimrod/G Interactions Grid Info servers Resource location Queuing System Process server Resource allocation (local) User process File access I/O server Gatekeeper node Job Wrapper Computational node Dispatcher Root node Scheduler Prmtc.. Engine Trade Server

A Nimrod/G Client CostDeadline Legion hosts Globus Hosts Bezek is in both Globus and Legion Domains

Change deadline/budget + Monitor activities

Adaptive Scheduling algorithms... LocateMachines DistributeJobs EstablishRates Meet requirements ? Deadlines and Budget Re-distributeJobs LocatemoreMachines

Nimrod/O Declarative Plan File Simulated Annealing Divide & Conquer Simplex P-BFGS Job Control Function Requests Function Values Nimrod or Clustor Super computer Cluster Jobs Results Nim Cache Function Requests Function Values Nimrod/O: Automatic Design Optimization  Search parameter space rather than exploring all options NimCache Nimrod/G Active Sheets - Excell cell func()s execution on the Grid

Related Work & Further Info.  AppLeS (UC. San Diego) – application level scheduling templates case-by-case for different Apps, soon PST.  NetSolve (UTK/ORNL) -- API for creating farms  Home, Distributed.net, ….  Millennium (UC. Berkeley) – remote execution environment on clusters and supports computational economy  CODINE/GRD (Genias/Gridware) – meets deadline by dominating over others share.  Mariposa- Distributed Database system (UC, Berkeley) – query with budget, creates sub-query & dividesbudget, trades with (remote) servers  More Info --