1. SDM Center A Quick Update on the TSI and PIW workflows SDM All Hands March 2-3, 2005 https://www-casc.llnl.gov/sdm/ Terence Critchlow, Xiaowen Xin, Bertram Ludaescher, Ilkay Altintas Mladen Vouk, Zengang Cheng, Lots of Others…. UCRL-PRES-209990

2. SDM Center Web page is up and running  Example workflows available for download  Documentation exists and is improving  List of (somewhat dated) publications https://www-casc.llnl.gov/sdm/

3. SDM Center TSI-1 Workflow Web interface for Simulation configuration submission Error notification Identify config files Are files consistent Move files to correct locations Submit batch request at NERSC Check on job statusDelay Generate directory listing Are there new files? Identify complete files Job still running? Count files Empty list Transfer files to HPSS Transfer completed correctly Transfer files to SB Transfer completed correctly Delete file Extract Get Variables Remap coordinates Create Chem vars Create neutrino vars Derive other varsWrite diagnostic file Generate plots Tool-1 Tool-2 Tool-3 Tool-4 Generate thumbnails Generate movie Update notebook Update web page Update notebook Update web page Delay Update notebook Update web page Update notebook Update web page

4. SDM Center Delay TSI-1 Workflow Web interface for Simulation configuration submission Error notification Identify config files Are files consistent Move files to correct locations Submit batch request at NERSC Check on job statusDelay Generate directory listing Are there new files? Identify complete files Job still running? Count files Empty list Transfer files to HPSS Transfer completed correctly Transfer files to SB Transfer completed correctly Delete file Extract Get Variables Remap coordinates Create Chem vars Create neutrino vars Derive other varsWrite diagnostic file Generate plots Tool-1 Tool-2 Tool-3 Tool-4 Generate thumbnails Generate movie Update notebook Update web page Update notebook Update web page Update notebook Update web page Update notebook Update web page Remote job submission - done

5. SDM Center TSI-1 Workflow Web interface for Simulation configuration submission Error notification Identify config files Are files consistent Move files to correct locations Submit batch request at NERSC Check on job statusDelay Generate directory listing Are there new files? Identify complete files Job still running? Count files Empty list Transfer files to HPSS Transfer completed correctly Transfer files to SB Transfer completed correctly Delete file Extract Get Variables Remap coordinates Create Chem vars Create neutrino vars Derive other varsWrite diagnostic file Generate plots Tool-1 Tool-2 Tool-3 Tool-4 Generate thumbnails Generate movie Update notebook Update web page Update notebook Update web page Delay Update notebook Update web page Update notebook Update web page Real-time file transfer - done

6. SDM Center TSI-1 Workflow Web interface for Simulation configuration submission Error notification Identify config files Are files consistent Move files to correct locations Submit batch request at NERSC Check on job statusDelay Generate directory listing Are there new files? Identify complete files Job still running? Count files Empty list Transfer files to HPSS Transfer completed correctly Transfer files to SB Transfer completed correctly Delete file Extract Get Variables Remap coordinates Create Chem vars Create neutrino vars Derive other varsWrite diagnostic file Generate plots Tool-1 Tool-2 Tool-3 Tool-4 Generate thumbnails Generate movie Update notebook Update web page Update notebook Update web page Delay Update notebook Update web page Update notebook Update web page Data analysis – not yet started

7. SDM Center TSI-1 Workflow  Developed modular, configurable, actors that can be used elsewhere  File filtering Use regular expressions to filter file list  File transfer User specified command transfers files between two remote locations  SSH Executes remote command, catches errors  Retry Repeat an action until successful or until user defined limit is reached

8. SDM Center Top level view of TSI workflow

9. SDM Center Submit actor Submit job using llsubmit Return the job number Debug actor lets you test without actually submitting job

10. SDM Center PIW Workflow  Added logging  Modularized workflow a bit more  Added debugging and testing code  New visualization actor

11. SDM Center Next Steps  Working through deadlock / race condition problems in Ptolemy PN director  Discovered because of test suite / debug actor  Need to be able to “merge” multiple streams  Data analysis section of TSI-1 workflow  More documentation

12. SDM Center Next talks  Steve Parker  SCIRun and SPA integration status  Mladen Vouk  On Large Data Flow Scientific Workflows: An Astropysics Case Study  Bertram Ludaescher  Kepler / SPA Extensions for Scientific Workflows – Now and Upcoming