1. ® OGC Workflow Activities to EC Workflows Community Group George Percivall and David Arctur Open Geospatial Consortium 4 June 2012 © 2012 Open Geospatial Consortium

2. OGC ® OGC presentation to EarthCube Workflows Community Group OGC members have developed and deployed operational workflow systems based on OGC and other standards, beginning in 2002. –http://www.opengeospatial.org/standardshttp://www.opengeospatial.org/standards –http://www.opengeospatial.org/projects/groups/workflowdwghttp://www.opengeospatial.org/projects/groups/workflowdwg Presentation that follows is organized using the EarthCube Workflow CG Questionnaire for the Community –Background Questions –General Workflow Questions –Advanced Workflow Questions –Questions on EarthCube Workflow WG Roadmap Activities, etc. © 2012 Open Geospatial Consortium2

3. OGC ® EarthCube Workflows Q&A #1 Background Questions: Improve efficiency of complicated computations Reproducibility Train new team members standard data products Document processing steps Multidisciplinary collaboration Scalability OGC Responses Context of workflow in OGC OGC Web Service standards Web Processing Service (WPS) for scalable interoperability to processing OGC Service Architecture AS Standard Topic 12 Service Chaining and Workflow © 2012 Open Geospatial Consortium3

4. © 2011 Open Geospatial Consortium OGC Web Services (OWS) Web Map Service (WMS) Web Map Tile Service (WMTS) Web Feature Service (WFS) Web Coverage Service (WCS) Catalogue (CSW) Geography Markup Language (GML) KML Others… The geospatial web is enabled by OGC standards: Relevant to geospatial applications: Critical Infrastructure, Emergency Management, Weather, Climate, Homeland Security, Defense & Intelligence, Oceans Science, etc Web Map Server Web Coverage Server Web Feature Server © 2012 Open Geospatial Consortium4

5. OGC Web Processing Service (WPS) WPS GetCapabilitiesExecuteDescribeProcess Algorithms Repository … … Algorithm 1 Data Handler Repository … … Data Handler A Communication over the web using HTTP WPS-client Web Processing Service © 2012 Open Geospatial Consortium 5

6. Design Patterns for Service Chaining OGC AS Topic 12 Service Architecture See also: “Chaining geographic information Web Services” N. Alameh, IEEE Internet Computing, Sept-Oct 2003, pp. 22-29. © 2012 Open Geospatial Consortium6

7. OGC ® EarthCube Workflows Q&A #2 and #3 General Workflow Questions describe one or more workflows that you use. What are the most difficult parts of your workflow models, processes, data available? On the Web? workflow technologies data preprocessing, data visualization, data reliability, QA/QC and uncertainty Interface to run workflows Advanced Workflow Questions Examples of workflow software What is most difficult about workflow technology today? What aspects find easy to support with workflows? Do you make your workflows available? On the Web? Lessons learned? exchanging workflows between systems find the data for your workflows in a convenient way main challenges in using workflows 10 years from now © 2012 Open Geospatial Consortium7 Response: Review of OWS Testbed implementations Summary of workflow implementations in OWS Testbeds Experience with workflow languages, Lessons learned

8. OGC ® Workflow for processing geo-information 8 … WCS (NASA Data Pool) WPS - Classification (Producer-C,Vendor-3) WPS - WCTS (Producer-B, Vendor-2) WFS (Producer-n, Vendor-x) Internet Hotspot Detection Service Chain OWS 2 Image Handling for Decision Support: Service Chaining with BPEL, OGC 04-078 OGC Interfaces Service chaining creates Value-added products Using web services, WSDL, BPEL Decision Support Client Geoprocessing Workflow developed in OGC Testbeds since 2004 – OWS-2 Assess Wildfire Activity © 2012 Open Geospatial Consortium

9. OWS-5 Conflation workflow architecture OWS-5 GeoProcessing Workflow Architecture Engineering Report, OGC 07-138r1 Conflation workflow in GMU’s BPEL designer tool WPS for Conflation Service and Quality Checking Rules Service accessed by WPS for processing rules Opaque chaining through WPS wrapper of BPEL Engine Lessons learned: Synchronous /Asyc, BPEL © 2012 Open Geospatial Consortium9

10. OGC ® OWS-5 Workflow Engine Interoperability Several workflow engines were used in OWS-5: BPEL, GeoBPMS, SensorML. Interoperability demonstrated using Workflow Chaining Services (WfCS) based on the WfMC Wf-XML © 2012 Open Geospatial Consortium10 OWS-5 GeoProcessing Workflow Architecture Engineering Report, Michael Werling OGC 07-138r1

11. OGC ® OWS-6 Workflow Progress Asynchronous Workflows & Security –Workflow Management Coalition (WfMC): WfXML, WfXML-R and XPDL –BPEL workflow language and associated workflow engine. Grid-Enabled Web Processing Service (WPS) Profiles –Open Grid Forum (OGF) specifications: HPC-Basic Profile, Simple API for Grid Applications (SAGA), Grid-RPC, Data Access and Integration set of specifications (WS-DAI-*), Web Services Resource Framework (WSRF) –encapsulating vs. integration Grid Computing WPS processes –WPS Trajectory Service process –WPS Plume Service process –WPS Rainfall Interpolation process –WPS Geophone Analysis process © 2012 Open Geospatial Consortium11

12. Using WPS to connect to Grid resources OWS-6 WPS Grid Processing Profile Engineering Report © 2012 Open Geospatial Consortium 12 The WPS Grid processing was done in OWS-6 Cloud computing now provides a similar resource for WPS-based computing

13. OWS-6 Debris flow monitoring system © 2012 Open Geospatial Consortium13

14. Debris flow operational scenario © 2012 Open Geospatial Consortium 14

15. OWS-7 Workflow for Internally Displaced Persons Camp Siting 1.Land Use Analysis 2.Exclude bad areas 3.Medical Facility Access 4.Food and Water Distribution Access 5.Transportation © 2012 Open Geospatial Consortium This workflow Accessed 6 remote OGC web services Created with ESRI Model Builder. Details in OGC 10-074 15

16. OGC ® EarthCube Workflows Q&A #4 EarthCube Workflow Working Group Roadmap Priorities: –Provenance, Scale, Reuse, Interoperability, Standards, User Interfaces, Semantic Metadata, Distributed Execution, Discovery, Data Integration, Validation, Automation, Tutorial Materials? Most critical items for the NSF EarthCube Workflow Roadmap? Appropriate points of contact? Involvement in future EarthCube Activity on Workflows? What incentives would you suggest? Other groups to engage? Educational resources? [G1] Synergies across groups OGC Response Interoperability, Standards, Interfaces Relation to science objectives of EarthCube OGC Workflow WG Workflow testbed activities, sponsored by NSF Best practices for workflow in Earth System Sciences Implementations in the community based on OGC © 2012 Open Geospatial Consortium16

17. From portal select desired theme and area of interest Wizard picks appropriate workflow for desired result Wizard Mozambique Disaster Management Information System (DMIS) Workflows Estimated rainfall accumulation and flood prediction model Flood Model Selected workflow automatically activates needed assets and models Baseline water level, flood waters and predicted flooding GEOSS AIP-2 flood prediction and response Led by NASA, Spot Image, Northrop Grumman, ERDAS © 2012 Open Geospatial Consortium 17

18. IP3 Client & Workflow engine IP3 Distributed Community Catalog/Mediator WCS - TOther Non-OGC ServicesGBIF Non-OGC Services OGC WPS Access to Model CSW GEOSS Portal WCSWFS req resp req resp req resp req resp req resp req resp Broker GEOSS AIP-2 biodiversity & climate change Led by CNR, Univ of Colorado, GBIF Research Scientist © 2012 Open Geospatial Consortium 18

19. OGC ® 52North tools at FOSS4G 2011 and beyond WPS client for Esri ArcMap –Able to encapsulate WPS processes in standard ArcMap geoprocessing tools –These tools (and underlying WPS processes) can be chained with ArcGIS ModelBuilder. –Data sources supported: zipped shapefiles (for vector) & geotiff (raster) –Tested stable with ArcGIS 10.0 Alternate approach: expose ArcGIS Server toolbox tools via WPS –(http://52north.org/communities/geoprocessing/wps/backends/52n-wps-ags.html) Prototypical web-based graphical modeling tool –Can chain different OGC services (WFS, WMS, WPS) –Needs updating to 52N WPS v3.0 –(http://geoprocessing.demo.52north.org:8080/workflowmodeller/)http://geoprocessing.demo.52north.org:8080/workflowmodeller/ Note: Esri ArcGIS Server 10.1 supports WPS (release due “mid June”) © 2012 Open Geospatial Consortium19

20. OGC ® OGC members workflow activities European Space Agency (ESA) Service Support Environment (SSE) –http://services.eoportal.org/http://services.eoportal.org/ –SSE service directory - basic and more complex Earth observation and GIS services.SSE service directory –"Join and Share Area” find partners boost development Grid-enabled Standard-compliant Open Computing Environment for Earth Science Exploration and Applications –George Mason University, Chen, A., L. Di, Y. Wei, Y. Bai, Y. Liu, 2006. in Proceedings of 2006 IGARSS06, July 31-August 04, 2006 Exposing the Kepler Scientific Workflow System as an OGC Web Processing Service –http://www.iemss.org/iemss2010/papers/S17/S.17.02.Exposing%20the%20Kepler%20Scientific%20Workflow%20System%2 0as%20an%20OGC%20Web%20Processing%20Service%20-%20BRADLEY%20LEE.pdfhttp://www.iemss.org/iemss2010/papers/S17/S.17.02.Exposing%20the%20Kepler%20Scientific%20Workflow%20System%2 0as%20an%20OGC%20Web%20Processing%20Service%20-%20BRADLEY%20LEE.pdf WPS orchestration using the Taverna workbench: The eScience approach, –In Computers & Geosciences, J. de Jesus, P. Walker, M. Grant, S. Groom, –http://www.sciencedirect.com/science/article/pii/S0098300411003906http://www.sciencedirect.com/science/article/pii/S0098300411003906 –WPS cookbook for the best practices wiki and have the pyWPS wiki (with some tutorial info) WPS and MyExperiment –http://www.myexperiment.org/tags/2024.htmlhttp://www.myexperiment.org/tags/2024.html © 2012 Open Geospatial Consortium20 © 2012 Open Geospatial Consortium20

21. Role of Workflow to Science Challenges to be met by EarthCube Relation of workflow to science objectives of EarthCube –Consider the topics in the Final Report of the Advisory Committee for Cyberinfrastructure, Task Force on Grand Challenges, NSF March 2011 Climate change mitigation strategies will require that workflows be run for regional adaptation strategies. Regional decision makers will need easy to use workflow to inform their strategies. Hazard Analysis and Management will benefit from workflow that readily and accurately uses archived and real-time sensing to inform the first responders and aid agencies Managing Greenhouse Gases will benefit from automated workflow procssesing of remote and in-situ measurements To meet these grand challenges, workflow will need to incorporate: –access to modeling and simulation systems, uncertainty quantification and propagation, integrated sensor-simulation systems, reproducibility, data infrastrcture, data-intensive science. © 2012 Open Geospatial Consortium21

22. OGC Workflow References OGC AS Topic 12 Service Architecture –http://www.opengeospatial.org/standards/ashttp://www.opengeospatial.org/standards/as OGC Web Processing Service Standard –http://www.opengeospatial.org/standards/wpshttp://www.opengeospatial.org/standards/wps –The WPS Standard provides rules for standardizing how inputs and outputs (requests and responses) for geospatial processing services. The standard also defines how a client can request the execution of a process, and how the output from the process is handled. It defines an interface that facilitates the publishing of geospatial processes and clients’ discovery of and binding to those processes. OGC Engineering Reports from OWS Testbeds –http://www.opengeospatial.org/standards/perhttp://www.opengeospatial.org/standards/per OGC Workflow Domain Working Group –http://www.opengeospatial.org/projects/groups/workflowdwghttp://www.opengeospatial.org/projects/groups/workflowdwg –The mission of the Workflow DWG is to establish a forum for describing, discussing, and solving any issues related to geospatial workflows. By geospatial workflow we mean any workflows that contain any or all processes that relate to geospatial processes and/or data. The primary focus of this DWG is to help individuals and organizations to identify smarter, easier, and more economical ways to build, migrate, manage, and maintain workflows. The intent of the Workflow DWG is not to discuss workflows in general, but rather how those workflows relate to geospatial information and address the special needs of the geospatial community. © 2012 Open Geospatial Consortium22