1. OGCE Workflow and LEAD Overview Suresh Marru, Marlon Pierce September 2009

2. Suresh Marru The LEAD Gateway

3. Suresh Marru LEAD Pioneering Technology

4. Suresh Marru Streaming Observations Storms Forming Forecast Model Data Mining Refine forecast grid Instrument Steering LEAD Dynamic Adaptive Infrastructure

5. Suresh Marru LEAD Scientists and Educational Interactions Developers Researcherss End Users Lowering the barrier for using complex end-to-end weather technologies Democratize Empower Facilitate

6. Suresh Marru Analyze & Predict Research & Reproducibility Education & Outreach Discover &Visualize

7. Suresh Marru Gateways/E-Science Community TG GIG Gateways (LEAD, GridChem, …) TeraGrid User Portal Generalize, Harden, Build Test Open Grid Computing Environments

8. Suresh Marru Middleware Services Compute & Data Resources User Interactions Computational Cloud GFac Services Local Lab Resources Other ClientsXBaya GUI XBaya Core Web Portal Computational Grids Event Bus Workflow Engine (ODE) XRegistry XMCCat Metadata Catalog Flexible Layered Service Oriented Architecture

9. Suresh Marru Application Wrapper Framework Scientific Applications are wrapped into web services by filling in a web- form. The Application Factory generates a web service for each application with I/O interfaces. Registers WSDL for the service with a registry Each service generates a stream of notifications that log the service actions back to the XMCCat Metadata Catalog, user monitoring, and provenance tracking toolscc Application Factory Run program & publish events App Service

10. Suresh Marru Application Wrapper Architecture

11. Suresh Marru Service Monitoring via Events The service output is a stream of events I am running your request I have started to move your input files I have all the files I am running your application The application is finished I am moving the output to you file space I am done These are automatically generated by the service using a distributed event system (WS-Eventing / WS-Notification) Topic based pub-sub system with a well known “channel” Application Service Instance Application Service Instance 1 1 2 2 3 3 4 4 5 5 6 6 Notification Channel Notification Channel listener publisher x x x x

12. Suresh Marru Workflow Suite Architecture

13. Suresh Marru Composition and Monitoring Abstract DAG Model BPEL 1.1 BPEL 2.0 SCUFL Python Dynamic Enactor/Interpreter Jython Based Enactor GPEL Engine Apache ODE Engine Taverna Python Runtime Message Bus Interoperable Xbaya Architecture

14. Suresh Marru 14 OVP/ RST/ MIG Fault Tolerance/ Recovery Service Application Performance Models Resource Reliability Models NWS, MDS BQP Deadline & Success Probability Portal BPEL Workflow Engine Application Service Notification Service Simple Recovery Architecture

15. Suresh Marru Workflow Composer

16. Suresh Marru Example: Assimilation Workflow

17. Suresh Marru Domain webs of large genomes – Input list of amino acid sequences – Identify all known domains – Construct webs Parallel processing Capability-type computing Ensemble-type processing (minimal network reqs) Capacity-type computing Science Independence Example

18. Suresh Marru Biology Gateway

19. Suresh Marru Polar Grid Example

20. Suresh Marru LEAD Lower entry barrier to using weather analysis tools Improve detection, analysis & prediction of mesoscale weather Motif-Network Transformation of sequenced genomes to “domain-space” Cyber-Infrastructure Evaluation Performance evaluation of future supercomputer architectures ADAM Algorithms for feature extraction, data normalization, classification and normalization GridChem Molecular Chemistry Grid helping researchers run chemistry applications on Grid Environment Example Gateways using OGCE Workflow Suite

21. Suresh Marru Live Demo & Questions?