Experiment Management with Microsoft Project Gregor von Laszewski Leor E. Dilmanian Acknowledgement: NSF NMI, CMMI, DDDAS

Outline Introduction Cyberaide.org Management Requirements of e-Science Microsoft Project for Managing Tasks Extending Microsoft Project for e-Science Conclusion References Acknowledgements

Introduction - Background E-Science is computationally or data intensive science, carried out in a distributed computing environment. Cyberinfrastructure is a research environment supporting advanced data acquisition and information processing services over the internet. Experiment Management is the management of a large number of experiments over Cyberinfrastructure

Introduction – Motivation There is a high entry barrier into Grid computing. Experiment Management on the Grid or Cyberinfrastructure is a complicated affair. Issues include: – Application Design – Scheduling of Large Scale Resources – Orchestration of Activity – Monitoring Execution and Quality of Service – Data Acquisition

e-Science Experiment Management von Laszewski’s Meta Computer Workflow & resource utilization, fault tolerance CoG Kit Workflow Abstractions CoG Kit Grid Ant CoG Experiment CoG Karajan Cyberaide Shell Project Experiment Management Super-Gramhttp://cyberaide.orghttp://cyberaide.org5 Queuing Systems Globus (Condor,Unicore, …) Social Networks Clouds

Cyberaidehttp://cyberaide.orghttp://cyberaide.org6 is a framework used to simplify Cyberinfrastructure development.

e-Science Requirements Focus on Science – Well known, familiar user interface – Easy to Navigate – Effectively manage large volumes of information – Advanced features: Time based workflows with soft deadlines. Calendar & resource views, toolbars, forms & graphical indicators… Integrated User Experience – Design Easily specify a distributed application. – Execution Seamless scheduling and execution on the Grid. – Monitoring Progress & Quality of Service. Integrated User Experience – Design Easily specify a distributed application. – Execution Seamless scheduling and execution on the Grid. – Monitoring Progress & Quality of Service.

Microsoft Project Is used for planning a project. Workflow or Project – a set of operations which contain the following: – Task – a unit of work. – Dependency – among two tasks, used to specify ordering. – Resource – people or machines used to carry out work. – Assignment /Mapping – of resources to tasks, to execute the workflow. A task becomes a job.

Microsoft Project Microsoft Project is used to by project managers to manage resources and tasks. Shown above: a Finance and Accounting System Implementation project. Source: Microsoft Project

Cyberaide Project Cyberaide contains a prototype called Cyberaide Project. – It uses the Microsoft Project software package as an elementary component. – Based on the methodology of project management software. – It is used to… Design a high level application. Execute the workflow on the Grid. Track progress and Quality of Service during execution.

Cyberaide Projecthttp://cyberaide.orghttp://cyberaide.org11 Console Window Proxy Initialization Task Table Gantt Chart

Technologies Visual Studio Professional 2008 C# and Visual Basic Microsoft Project 2007 Professional Microsoft Project Primary Interoperability Assembly Java CoG Kit – Cyberaide – in the near future, Cyberaide will the functionality the Java Cog Kit currently contains, but in additional programming languages. Apache CLI Cyberaide Shell Mediator – in the near future, this is how interaction with the Grid and Clusters will be working. WSI – web service interoperability

Cyberaide Project The CyberaideProject Class Written in C# Console Application Execution Shell Runs user commands The Command Interface Specify a command Register with the Shell Uses Apache CLI for parsing Manipulates instance of MSP The CyberaideProject Class Written in C# Console Application Execution Shell Runs user commands The Command Interface Specify a command Register with the Shell Uses Apache CLI for parsing Manipulates instance of MSP Registers/Executes Console Application Launches Microsoft Project Accepts Commands Console Application Launches Microsoft Project Accepts Commands Command Interface Apache CLI Manipulate MSP Command Interface Apache CLI Manipulate MSP

Command Interfacehttp://cyberaide.orghttp://cyberaide.org14

Commands A set of commands allow the user to manipulate, execute, and manage workflows. Commands fall into the categories of Workflow Manipulation, Workflow Execution, and Utility Functions. A set of commands allow the user to manipulate, execute, and manage workflows. Commands fall into the categories of Workflow Manipulation, Workflow Execution, and Utility Functions. CommandDescriptionCommand Type taskAdd a Task.Workflow Manipulation mapMap Task/Resource.Workflow Execution delDelete a Task.Utility Function editEdit Task Information. depAdd a Task Dependency. rdepRemove a Task Dependency. resAdd a Resource. run-allRun a Workflow. saveSave a Workflow in JSON. loadLoad a Workflow in JSON. listList all Tasks with a valid name. findFind a Task by its name.

Cyberaide Integration An easy client interface provides access through a mediator service to a variety of different services. MS Project Ruby Python Java Mediator & Abstractions TeraGrid Amazon Cloud Amazon Cloud facebook iGoogle Social Networks Social Networks Globus GPIR, NWS, … Cloud Grid Client Services JavaScript C#

Loadinghttp://cyberaide.orghttp://cyberaide.org17

Design a Projecthttp://cyberaide.orghttp://cyberaide.org18

Execute & Monitorhttp://cyberaide.orghttp://cyberaide.org19

Task Outputhttp://cyberaide.orghttp://cyberaide.org20

Project Completionhttp://cyberaide.orghttp://cyberaide.org21

Summary Microsoft Project: Familiarity, Usability, Productivity – Task Table in Spreadsheet Format – Easy Navigation & Viewing Resource Mapping -> Jobs Integration with TeraGrid – through CoG Kit to TeraGrid – We have are developing a Grid mediator (e.g. a Super Gram Service)

Conclusion Cyberaide Project can be used to design applications in e- Science – Supports the process of developing, executing and monitoring workflows for the Grid Familiarity, simplicity and user friendliness are the major considerations – Useful for the casual Grid user familiar with or using Microsoft Project Spend less time learning software, and more time making discovery – Increases productivity

References [1] von Laszewski, G., & Dilmanian, L. E-Science Project and Experiment Management with Microsoft Project. Accepted for publication at GCE08 at SC08. [2] G. von Laszewski, J. Gawor, P. Lane, N. Rehn, M. Russell, and K. Jackson, “Features of the Java Commodity Grid Kit,” Concurrency and Computation: Practice and Experience, vol. 14, pp. 1045–1055, [Online]. Available: [3] G. von Laszewski, B. Ruscic, K. Amin, P. Wagstrom, S. Krishnan, and S. Nijsure, “A Framework for Building Scientific Knowledge Grids Applied to Thermochemical Tables,” The International Journal of High Performance Computing Applications, vol. 17, no. 4, pp. 431– 447, Winter [Online]. Available: [4] G. von Laszewski, “Java CoG Kit Workflow Concepts,” Journal of Grid Computing, Jan. 2006, 5. [Online]. Available: [5] G. von Laszewski, A. J. Younge, X. He, and F. Wang, “GridShell: Interactive Task Management for Grid and Cluster Computing,” (im preparation for review), Nov

Acknowledgements NSF DDDAS NSF CMMI NSF SDCI NMI