Integrated Scientific Workflow Management for the Emulab Network Testbed Eric Eide, Leigh Stoller, Tim Stack, Juliana Freire, and Jay Lepreau and Jay Lepreau University of Utah, School of Computing USENIX 2006 / June 3, 2006

2 This Talk in One Slide Current network testbeds Current network testbeds …manage the “laboratory” …manage the “laboratory” …not the experimentation process. …not the experimentation process. → A big problem for large-scale activities! → A big problem for large-scale activities! Evolve Emulab for experiments based on scientific workflows Evolve Emulab for experiments based on scientific workflows Big mutual benefits: testbed ↔ workflow Big mutual benefits: testbed ↔ workflow Work in progress Work in progress

3 Example: UAV Simulation A distributed, real-time application A distributed, real-time application Evaluate improvements to real-time middleware Evaluate improvements to real-time middleware vs. CPU load vs. CPU load vs. network load vs. network load 4 research groups 4 research groups x 19 experiments x 19 experiments x 56 metrics x 56 metrics UAV Receiver ATR images → ← images alerts →

4 Use Emulab ConceptExperimentEmulate write “ns” file “swap in”

5 Problems Solved I get machines! I get machines! 328 PCs, and more 328 PCs, and more Time- & space-shared Time- & space-shared Loads OS and software Loads OS and software I get network! I get network! Config. topology & quality Config. topology & quality I get to collaborate! I get to collaborate! Available to researchers and educators worldwide Available to researchers and educators worldwide File storage, , … File storage, , …

6 Problems Not Solved “Now what?” Getting off the ground Run all my software Add instrumentation Collect all my data Analyze it Scaling up 19 configurations Automation

7 More Problems Not Solved “How did I get here?” Over the short term… “Where are the results I got last week?” “How did I get those results anyway?” “What if…?” …and the long term Reproducing results Reusing artifacts

8 Idea: Scientific Workflow Managing activities, inputs, and outputs is the job of a scientific workflow system Managing activities, inputs, and outputs is the job of a scientific workflow system Our approach: evolve Emulab with integrated support for scientific workflows Our approach: evolve Emulab with integrated support for scientific workflows Build on existing abstractions & mechanisms Build on existing abstractions & mechanisms Resource focus → user & task focus Resource focus → user & task focus Users work “within” and “across” experiments Users work “within” and “across” experiments

9 Contributions Address demand + opportunity Address demand + opportunity Users need to manage large-scale complexity Users need to manage large-scale complexity A symbiotic combination: leverage and impact A symbiotic combination: leverage and impact Advance the applicability of testbeds Advance the applicability of testbeds Not just Emulab — e.g., PlanetLab and DETER Not just Emulab — e.g., PlanetLab and DETER Advance scientific workflow systems Advance scientific workflow systems Exploit testbed capabilities — e.g., “total control” Exploit testbed capabilities — e.g., “total control” Address testbed requirements — e.g., flexible use Address testbed requirements — e.g., flexible use

10 Issue: Encapsulation Current “experiment” model is not fully encapsulating Current “experiment” model is not fully encapsulating Topology + static events Topology + static events Need everything else! Need everything else! Challenge: specification Challenge: specification Complete and precise… Complete and precise… …w/o huge user burden …w/o huge user burden Approach: be automatic Approach: be automatic E.g., track files used E.g., track files used Snapshot, archive, restore Snapshot, archive, restore User can refine “extent” User can refine “extent” ns file OSespackages my software inputsoutputs NFS monitors packet monitors AJAX GUI Subversion repo. datapository (DB) research filesystems

11 Issue: Definition vs. Execution Current “experiment” has multiple roles Definition The thing that you run Challenge: representing relationships Multiple runs of one setup Similar configurations Approach: a new model of experimentation Separate the roles Evolve the new abstractions

12 New Model Template Template Swapin Swapin Experiment Experiment Activity Activity Record Record n = 2 n = 4

13 Issue: History Research and educational plans are dynamic By design & by discovery Challenge: safe exploration Fork Back up Approach: keep history & support temporal navigation Keep template revisions Track provenance Locate, repeat, and reuse rev 1.1 bigger nets add params oops: need new measurements what about loss?

14 Implementation in Progress Definition Execution & History Data Analysis

15 Conclusion Large and powerful testbeds Large and powerful testbeds …enable complex and large-scale activities …enable complex and large-scale activities …lead to complex and large-scale workflow management problems …lead to complex and large-scale workflow management problems Integrated workflow management can leverage the strengths of testbeds Integrated workflow management can leverage the strengths of testbeds Systems approach — and systems challenges Systems approach — and systems challenges → Better testbeds and workflow systems → Better testbeds and workflow systems

Thanks!

17 Extra Slides After This Point