1. Nimrod & NetSolve Sathish Vadhiyar

2. Nimrod Sources/Credits: Nimrod web site & papers

3. Background For execution of parametric experiments across distributed computers User describes plan file that declares parameters Parametric studies – range of different simulations calculated using the same program Need for a Grid 3 variables, 4 values – 64 experiments 3 variables, 4 values – 64 experiments Each experiment – several hours Each experiment – several hours

4. Sample plan file parameter iseed integer range from 100 to 4000 step 100; parameter thick label "BUC thickness" float range from 1.1 to 2.0 step 0.1; parameter jseed integer compute thick*1000; task nodestart copy ccal.$OS node:./ccal copy ccal.$OS node:./ccal copy dummy node:. copy dummy node:. copy ccal.dat node:. copy ccal.dat node:. copy skel.inp node:. copy skel.inp node:.endtask task main node:substitute skel.inp ccal.inp node:substitute skel.inp ccal.inp node:execute./ccal node:execute./ccal copy node:ccal.op ccalout.$jobname copy node:ccal.op ccalout.$jobnameendtask

5. Phases of Computational Experiment 1.Experiment pre-processing, when data is set up for the experiment; 2.Execution pre-processing, when data is prepared for a particular execution; 3.Execution, when the program is executed for a given set of parameter values; 4.Execution post-processing, when data from a particular execution is reduced; 5.Experiment post-processing, when results are processed, for example by running data interpretation or visualization software.

6. Illustration

7. Nimrod Architecture

8. Architecture

9. Architecture Components Client Client Parametric engine Parametric engine Scheduler Scheduler Dispatcher Dispatcher Job wrapper Job wrapper

10. Components Parametric engine Persistent job service Persistent job service Interacts with the client, schedule advisor and dispatcher Interacts with the client, schedule advisor and dispatcher Takes declarative plan from the user Takes declarative plan from the userScheduler Objectives – meet deadlines, minimize cost Objectives – meet deadlines, minimize costDispatcher Starts remote component called job wrapper Starts remote component called job wrapper Updates status of task to parametric engine Updates status of task to parametric engine Job wrapper Responsible for staging-in, execution and staging out Responsible for staging-in, execution and staging out

11. Cost Model Cost / Priority matrix defined based on specification by resource providers Nimrod/G scheduler performs discovery and allocation of resources based on specified execution times and cost constraints Cost of experiment varies depending on the load

12. Scheduling Heuristic 1.Discovery 1. Initial filtering of resources based on cost specifications 2. Identification of lowest-cost set of resources able to meet deadlines 2.Allocations 1. Jobs allocated from the queue to the resources identified in step 1 3.Monitoring 1. Completion time of jobs monitored 2. Execution rate established 4.Refinement 1. Execution rate used to update expected completion times of remaining jobs 2. Revisit steps 1 and 2

13. Experiments

14. Ionization chamber calibration Chamber response to front wall thickness ion-pair = 400 tasks Each model involved about 40 minutes – 140 minutes 3 experiments – 10-hr, 15-hr, 20-hr deadline

15. No. of resources Vs time

16. Cost vs Time

20. Another Experiment

21. Experimental setup 165 cpu jobs, each 5 min. in duration Deadline – 2 hours Budget - 396000 2 strategies: Optimize time Optimize time Optimize cost Optimize cost

22. Results

23. Time Optimization

24. Cost Optimization

25. Scheduling Adaptive scheduling algorithms Time minimization and limited budget (etime optimal) Time minimization and limited budget (etime optimal) Time minimization and unlimited budget (etime highoptimal) Time minimization and unlimited budget (etime highoptimal) Cost minimization and limited by deadline (ecost optimal) Cost minimization and limited by deadline (ecost optimal) None minimization, limited time and cost (etime + ecost optimal) None minimization, limited time and cost (etime + ecost optimal)

26. Nimrod / O Optimization of parameters to minimize objective function Case study: optimize shape and angle of attack of airfoil that maximizes the lift to drag ratio Design optimization problem Objective function can be non-linear, contain noise, can be continuous or discrete No single optimization algorithm can give the best result Nimrod / O supports a range of algorithms

27. Contd … Search algorithms P-BFGS P-BFGS Simplex Simplex Divide-and-conquer Divide-and-conquer Simulated annealing Simulated annealing

28. Plan file modified by Nimrod / O

29. References Abramson D, Lewis A, Peachey T, Fletcher, C., “An Automatic Design Optimization Tool and its Application to Computational Fluid Dynamics”, SuperComputing 2001, Denver, Nov 2001. Abramson, D., Sosic, R., Giddy, J., Cope, M. "The Laboratory Bench: Distributed Computing for Parametised Simulations", 1994 Parallel Computing and Transputers Conference, Wollongong, Nov 94, pp 17 27. Abramson D., Sosic R., Giddy J. and Hall B., "Nimrod: A Tool for Performing Parametised Simulations using Distributed Workstations", The 4th IEEE Symposium on High Performance Distributed Computing, Virginia, August 1995.

30. References Abramson, D., Giddy, J. and Kotler, L. High Performance Parametric Modeling with Nimrod/G: Killer Application for the Global Grid?, International Parallel and Distributed Processing Symposium (IPDPS), pp 520- 528, Cancun, Mexico, May 2000. Buyya, R., Abramson, D. and Giddy, J. Nimrod/G: An Architecture of a Resource Management and Scheduling System in a Global Computational Grid, HPC Asia 2000, May 14-17, 2000, pp 283 289, Beijing, China. Abramson, D, Buuya, R. and Giddy, J. “A Computational Economy for Grid Computing and its Implementation in the Nimrod-G Resource Broker”, Future Generation Computer Systems. Volume 18, Issue 8, Oct-2002. Buyya, R., Giddy, J. and Abramson, D. "An Evaluation of Economy- based Resource Trading and Scheduling on Computational Power Grids for Parameter Sweep Applications", Workshop on Active Middleware Services (AMS 2000), (in conjuction with Ninth IEEE International Symposium on High Performance Distributed Computing), Kluwer Academic Press, August 1, 2000, Pittsburgh, USA.

31. Junk !!

32. Nimrod Architecture

33. Components Generator Input: plan file Input: plan file Processes plan file, gives choices to the user regarding parameters Processes plan file, gives choices to the user regarding parameters Output: run file (description of a job) Output: run file (description of a job)Dispatcher Input: run file Input: run file Stages file to remote resources Stages file to remote resources Runs jobs on remote resources Runs jobs on remote resources

34. Nimrod-G Architecture Origin: Implements scheduling and monitoring Exists for the entire duration of the experiment Responsible for execution of experiment within specified time and cost constraints Client User interacts with the Origin process through the client Multiple clients can connect to a single origin process and monitor the same experiment

35. Nimrod Components Nimrod Resource Broker Origin process spawns NRB on the remote site Origin process spawns NRB on the remote site Interacts with GRAM Interacts with GRAM Capabilities beyond GRAM including file staging, creation of jobs and process control Capabilities beyond GRAM including file staging, creation of jobs and process control

36. experiments 90-second jobs over 10 simulated queues with different access costs (Q1=10, Q2 = 12 etc.) 100 jobs, 9000 seconds 10 queues, 900 seconds optimal Deadlines – 990, 1980, 2970 Costs – 252000, 171000, 126000