Batch Queue Prediction Warren Smith

12/1/2009 TeraGrid Quarterly - Batch Queue Prediction2 Recommendation Scheduling Working Group recommends that TeraGrid deploys Karnak Current QBETS deployment not sustainable –Service deployed outside TeraGrid Cannot be moved onto TeraGrid –Source code unavailable –Licensing not known –Developer focus is elsewhere (Eucalyptus) Working group evaluated Karnak and Moab –Also discussed Catalina as an alternative to Moab –Full report (draft) linked from sched-wg page ng_WG

12/1/2009 TeraGrid Quarterly - Batch Queue Prediction3 Karnak Advantages Cost –Kanak is free, Moab has a cost At least cost of the TeraGrid Moab contract (not currently being used) Additional cost, if more sockets needed than in contract License –Karnak is open source - Apache 2 –Moab is commercial - access to source code is possible, but only while an agreement continues Capabilities –Karnak provides information on expected accuracy of it’s predictions, Moab does not –Karnak provides historical statistics about jobs Prediction accuracy –Karnak is currently slightly better, and is expected to improve RP effort –For Karnak, RPs just install GLUE 2 information provider –For Moab, RPs without Moab install/configure/maintain Moab in monitor mode –For Moab, RPs would install a new Moab prediction service Time to deploy –Karnak should be faster to deploy Prototype already available - Requires less effort from RPs than deploying Moab in monitor mode Doesn’t require any additional contract negotiations

12/1/2009 TeraGrid Quarterly - Batch Queue Prediction4 Moab Advantages Stability –Moab is production, commercial code –Karnak is research software on the way to production Support –Moab has a development & support team –Karnak has a single developer (Warren) Documentation –Moab has lots –Karnak documentation in progress Familiarity –Moab is familiar to a number of TeraGrid RPs

12/1/2009 TeraGrid Quarterly - Batch Queue Prediction5 Preliminary Tasks Finalize report (12/11/2009) Provide new GLUE 2 version (RP testing 12/7/2009) –Publishes job information to secure MDS –Port to additional schedulers Moab (complete) Condor (complete) Cobalt (in progress) PBS at PSC (jobs ordered by scheduling priority)

12/1/2009 TeraGrid Quarterly - Batch Queue Prediction6 Karnak Tasks Publish REST interface description (1/8/2010) Initial optimization of predictions for TeraGrid (1/15/2010) Provide client programs (1/22/2010) Publish user documentation (2/5/2010)

12/1/2009 TeraGrid Quarterly - Batch Queue Prediction7 Usage-Related Tasks Integrate into TeraGrid user portal (2/2010?) Integrate into metaschedulers (e.g. Condor-G matchmaking) (2/13/2010) –Condor-G class ads created from GLUE 2 –Karnak predictions created from GLUE 2