1. Experiment Management with Microsoft Project Gregor von Laszewski Leor E. Dilmanian http://www.cyberaide.org Acknowledgement: NSF NMI, CMMI, DDDAS http://cyberaide.org1

2. Outline Introduction Cyberaide.org Management Requirements of e-Science Microsoft Project for Managing Tasks Extending Microsoft Project for e-Science Conclusion References Acknowledgements http://cyberaide.orghttp://cyberaide.org2

3. Introduction - Background E-Science is computationally or data intensive science, carried out in a distributed computing environment. Cyberinfrastructure is a research environment supporting advanced data acquisition and information processing services over the internet. Experiment Management is the management of a large number of experiments over Cyberinfrastructure http://cyberaide.orghttp://cyberaide.org3

4. Introduction – Motivation There is a high entry barrier into Grid computing. Experiment Management on the Grid or Cyberinfrastructure is a complicated affair. Issues include: – Application Design – Scheduling of Large Scale Resources – Orchestration of Activity – Monitoring Execution and Quality of Service – Data Acquisition http://cyberaide.orghttp://cyberaide.org4

5. e-Science Experiment Management von Laszewski’s Meta Computer Workflow & resource utilization, fault tolerance CoG Kit Workflow Abstractions CoG Kit Grid Ant CoG Experiment CoG Karajan Cyberaide Shell Project Experiment Management Super-Gramhttp://cyberaide.orghttp://cyberaide.org5 Queuing Systems Globus (Condor,Unicore, …) Social Networks Clouds

6. Cyberaidehttp://cyberaide.orghttp://cyberaide.org6 is a framework used to simplify Cyberinfrastructure development.

7. e-Science Requirements Focus on Science – Well known, familiar user interface – Easy to Navigate – Effectively manage large volumes of information – Advanced features: Time based workflows with soft deadlines. Calendar & resource views, toolbars, forms & graphical indicators… Integrated User Experience – Design Easily specify a distributed application. – Execution Seamless scheduling and execution on the Grid. – Monitoring Progress & Quality of Service. Integrated User Experience – Design Easily specify a distributed application. – Execution Seamless scheduling and execution on the Grid. – Monitoring Progress & Quality of Service. http://cyberaide.orghttp://cyberaide.org7

8. Microsoft Project Is used for planning a project. Workflow or Project – a set of operations which contain the following: – Task – a unit of work. – Dependency – among two tasks, used to specify ordering. – Resource – people or machines used to carry out work. – Assignment /Mapping – of resources to tasks, to execute the workflow. A task becomes a job. http://cyberaide.orghttp://cyberaide.org8

9. Microsoft Project Microsoft Project is used to by project managers to manage resources and tasks. Shown above: a Finance and Accounting System Implementation project. http://cyberaide.orghttp://cyberaide.org9 Source: Microsoft Project

10. Cyberaide Project Cyberaide contains a prototype called Cyberaide Project. – It uses the Microsoft Project software package as an elementary component. – Based on the methodology of project management software. – It is used to… Design a high level application. Execute the workflow on the Grid. Track progress and Quality of Service during execution. http://cyberaide.orghttp://cyberaide.org10

11. Cyberaide Projecthttp://cyberaide.orghttp://cyberaide.org11 Console Window Proxy Initialization Task Table Gantt Chart

12. Technologies Visual Studio Professional 2008 C# and Visual Basic Microsoft Project 2007 Professional Microsoft Project Primary Interoperability Assembly Java CoG Kit – Cyberaide – in the near future, Cyberaide will the functionality the Java Cog Kit currently contains, but in additional programming languages. Apache CLI Cyberaide Shell Mediator – in the near future, this is how interaction with the Grid and Clusters will be working. WSI – web service interoperability http://cyberaide.orghttp://cyberaide.org12

13. Cyberaide Project The CyberaideProject Class Written in C# Console Application Execution Shell Runs user commands The Command Interface Specify a command Register with the Shell Uses Apache CLI for parsing Manipulates instance of MSP The CyberaideProject Class Written in C# Console Application Execution Shell Runs user commands The Command Interface Specify a command Register with the Shell Uses Apache CLI for parsing Manipulates instance of MSP http://cyberaide.orghttp://cyberaide.org13 Registers/Executes Console Application Launches Microsoft Project Accepts Commands Console Application Launches Microsoft Project Accepts Commands Command Interface Apache CLI Manipulate MSP Command Interface Apache CLI Manipulate MSP

14. Command Interfacehttp://cyberaide.orghttp://cyberaide.org14

15. Commands A set of commands allow the user to manipulate, execute, and manage workflows. Commands fall into the categories of Workflow Manipulation, Workflow Execution, and Utility Functions. A set of commands allow the user to manipulate, execute, and manage workflows. Commands fall into the categories of Workflow Manipulation, Workflow Execution, and Utility Functions. http://cyberaide.orghttp://cyberaide.org15 CommandDescriptionCommand Type taskAdd a Task.Workflow Manipulation mapMap Task/Resource.Workflow Execution delDelete a Task.Utility Function editEdit Task Information. depAdd a Task Dependency. rdepRemove a Task Dependency. resAdd a Resource. run-allRun a Workflow. saveSave a Workflow in JSON. loadLoad a Workflow in JSON. listList all Tasks with a valid name. findFind a Task by its name.

16. Cyberaide Integration An easy client interface provides access through a mediator service to a variety of different services. http://cyberaide.orghttp://cyberaide.org16 MS Project Ruby Python Java Mediator & Abstractions TeraGrid Amazon Cloud Amazon Cloud facebook iGoogle Social Networks Social Networks Globus GPIR, NWS, … Cloud Grid Client Services JavaScript C#

17. Loadinghttp://cyberaide.orghttp://cyberaide.org17

18. Design a Projecthttp://cyberaide.orghttp://cyberaide.org18

19. Execute & Monitorhttp://cyberaide.orghttp://cyberaide.org19

20. Task Outputhttp://cyberaide.orghttp://cyberaide.org20

21. Project Completionhttp://cyberaide.orghttp://cyberaide.org21

22. Summary Microsoft Project: Familiarity, Usability, Productivity – Task Table in Spreadsheet Format – Easy Navigation & Viewing Resource Mapping -> Jobs Integration with TeraGrid – through CoG Kit to TeraGrid – We have are developing a Grid mediator (e.g. a Super Gram Service) http://cyberaide.orghttp://cyberaide.org22

23. Conclusion Cyberaide Project can be used to design applications in e- Science – Supports the process of developing, executing and monitoring workflows for the Grid Familiarity, simplicity and user friendliness are the major considerations – Useful for the casual Grid user familiar with or using Microsoft Project Spend less time learning software, and more time making discovery – Increases productivity http://cyberaide.orghttp://cyberaide.org23 laszewski@gmail.com

24. References [1] von Laszewski, G., & Dilmanian, L. E-Science Project and Experiment Management with Microsoft Project. Accepted for publication at GCE08 at SC08. [2] G. von Laszewski, J. Gawor, P. Lane, N. Rehn, M. Russell, and K. Jackson, “Features of the Java Commodity Grid Kit,” Concurrency and Computation: Practice and Experience, vol. 14, pp. 1045–1055, 2002. [Online]. Available: http://www.mcs.anl.gov/gregor/papers/vonLaszewski--cog-features.pdf http://www.mcs.anl.gov/gregor/papers/vonLaszewski--cog-features.pdf [3] G. von Laszewski, B. Ruscic, K. Amin, P. Wagstrom, S. Krishnan, and S. Nijsure, “A Framework for Building Scientific Knowledge Grids Applied to Thermochemical Tables,” The International Journal of High Performance Computing Applications, vol. 17, no. 4, pp. 431– 447, Winter 2003. [Online]. Available: http://www.mcs.anl.gov/gregor/papers/vonLaszewski--knowledge-grid.pdfhttp://www.mcs.anl.gov/gregor/papers/vonLaszewski--knowledge-grid.pdf [4] G. von Laszewski, “Java CoG Kit Workflow Concepts,” Journal of Grid Computing, Jan. 2006, http://dx.doi.org/10.1007/s10723-005-9013- 5. [Online]. Available: http://www.mcs.anl.gov/gregor/papers/vonLaszewski-workflow-taylor-anl.pdfhttp://www.mcs.anl.gov/gregor/papers/vonLaszewski-workflow-taylor-anl.pdf [5] G. von Laszewski, A. J. Younge, X. He, and F. Wang, “GridShell: Interactive Task Management for Grid and Cluster Computing,” (im preparation for review), Nov. 2008. http://cyberaide.orghttp://cyberaide.org24 laszewski@gmail.com

25. Acknowledgements NSF DDDAS NSF CMMI 0540076 NSF SDCI NMI 0721656 http://cyberaide.orghttp://cyberaide.org25