Workshop on Workflows on Earth Observation Steven Ramage, OGC 21 June 2010, Sir Clive Granger Building, University of Nottingham

Making location count Copyright © 2010, Open Geospatial Consortium OGC workflow areas OWS-8 Call ends this week Decision Fusion, as described in the RFI, provides analysts an environment of interoperable services for situation assessment, impact assessment and decision support, based on information from multiple sensors and databases, e.g., multi-INT sources. The study includes recent advances such as social networking for decision fusion. Though the focus of the study is on military intelligence (“INT”), decision fusion is equally relevant to business intelligence, urban planning, and many other domains. Domain Working Groups –Earth system science –Emergency response and disaster management –Meteorology and oceans –Workflow

Making location count Copyright © 2010, Open Geospatial Consortium The importance of workflow Workflows are important because real jobs rarely involve few, simple tasks. Creating value in almost any area of endeavor requires the sequencing and organisation of human efforts, or of digital or mechanical processes, e.g. –Supply chain manufacturing –Insurance claims processing –Building design All can be studied as workflows. The geospatial realm is no different. Image processing and spatial modeling can also be characterized as workflows.

Making location count Copyright © 2010, Open Geospatial Consortium Description of workflow In information systems workflows are a series of coordinated analytical and information processing steps Typically to transform data into information or knowledge Workflows could be described as Web-based scripts or macros that automate serial information tasks.

Making location count Copyright © 2010, Open Geospatial Consortium OGC Interoperability Program Collaborative, standards-based interoperability initiatives Sponsors define operational issues where standards can help and initiatives are created Technology providers come together to prototype solutions to tackle sponsors’ interoperability problems. OGC Web Services (OWS) initiatives underway for years OWS-5 had geospatial workflows as a primary focus, a detailed video report is available at: Leads to the OGC Specification Program and then the Outreach and Community Adoption Program

Making location count Copyright © 2010, Open Geospatial Consortium OGC Web Services standards OGC has four main OGC Web services standards for geospatial data sharing and processing: 1.OGC Web Mapping Service Interface Standard (WMS) defines a Web API for requesting a picture of data, such as a PNG, JPEG, or GIF. 2.OGC Web Feature Service Interface Standard (WFS) defines a Web API for accessing raw vector data in formats such as GML, KML or GeoRSS and adding, updating and deleting data.

Making location count Copyright © 2010, Open Geospatial Consortium OGC Web Services standards OGC has four main OGC Web services standards for geospatial data sharing and processing: 3.OGC Web Coverage Service Interface Standard (WCS) defines a web API for accessing raster data in formats such as GeoTIFF, JPEG2000, or HDF WCS also supports adding, updating and deleting data. 4.OGC Web Processing Service Interface Standard (WPS) defines a web API for running an algorithm or model, specifying data inputs and outputs

Making location count Copyright © 2010, Open Geospatial Consortium “Chaining” Web Services for decision support … WCS WPS – Classification WPS - WCTS WFS Internet Web servers OGC Interfaces Service chaining creates value-added products Decision Support Client Geoprocessing worklow developed in OGC testbeds since 2004 Assess Wildfire Activity

Making location count Copyright © 2010, Open Geospatial Consortium BPEL workflow in OWS-5 Participants chained combinations of services together to address the requirements of geospatial data conflation and image processing. It also addressed an important gap, which was the lack of Web service communication - SOAP (Simple Object Access Protocol) and descriptions - WSDL (Web Service Description Language). The OGC and the International Organisation for Standardisation, Technical Committee 211 (ISO/TC211; provide open specifications for geo- Web services and W3C ( has developed protocols for SOAP and WSDL.

Making location count Copyright © 2010, Open Geospatial Consortium BPEL workflow in OWS-5 Both W3C standards are critical for creating and managing workflows using industry standards such as OASIS’s BPEL (Business Process Execution Language). BPEL is the workflow description language used in many end-user service chaining tools such as Oracle BPEL Designer and George Mason University’s BPEL Power.

Making location count Copyright © 2010, Open Geospatial Consortium Conflation workflow architecture

Making location count Copyright © 2010, Open Geospatial Consortium Conflation workflow architecture The OWS-5 geo-processing workflow thread bound together WFS and WPS services in BPEL scripts to model data conflation in a consistent, repeatable fashion. This successful prototype signaled the utility of this approach for a host of geospatial tasks that require the combination of many different geospatial services and operations. Fusion activities developed in OWS-7 and will have more focus in this area for OWS-8.

Making location count Copyright © 2010, Open Geospatial Consortium OGC Web Processing Service (WPS) WPS GetCapabilitiesExecuteDescribeProcess Algorithms Repository … … Algorithm 1 Data Handler Repository … … Data Handler A Communication over the web using HTTP WPS-client Web Processing Service

Making location count Copyright © 2010, Open Geospatial Consortium REST-oriented workflow in OWS-5 The SOAP/WSDL/BPEL approach to chaining Web services is one of the most widely used ways to implement service oriented architectures (SOA). However, other approaches, notably REST (Representational State Transfer) services, are also used to implement services in an enterprise. Some feel that Web services do not need the extra layer of description provided by SOAP and WSDL to build reliable workflows.

Making location count Copyright © 2010, Open Geospatial Consortium REST uses HTTP and other parts of the basic Web platform. In the SWE (Sensor Web Enablement) thread of OWS-5, participants used a REST-based architecture and WfXML to create workflows that allowed raw earth observation data to be acquired from assets, such as the EO-1 satellite, and then utilized by different WPS. Relevant data were then published and distributed to end-users in formats such as GeoAtom (Atom with GeoRSS extensions) and KML. REST-oriented workflow in OWS-5

Making location count Copyright © 2010, Open Geospatial Consortium Integrating smoke WPS into WfXML-R workflow GMU GRASS SOAP/WSDL GRASS Image Algebra EO-1 SOS Google Earth Execute Inputs: URL Grid 1 URL Grid 2 OutputType EO Sensor Web Workflow GMU WCS-T GetCapabilities DescribeProcess NGC EO-1 Smoke WPS Calculated smoke geotiff Classified smoke geotiff Call calculation Call classifier Processes: FineSmoke CoarseSmoke Inputs: URL Grid 1 URL Grid 2 OutputType Output: URL Grid 3

OWS-6 WPS Grid Processing Bastian Baranski and Bastian Schäffer, Univ. Muenster Institute for Geoinformatics (IfGI) and 52° North, Germany Andrew Woolf, Science and Technology Facilities Council (STFC), UK Lan-Kun Chung, GIS Center, Feng Chia University, Taiwan

Making location count Copyright © 2010, Open Geospatial Consortium OWS-6 WPS grid processing Grid processing profiles of WPS - specification –WPS grid processing profile integrated with grid computing infrastructure: Job Submission Description Language (JSDL) High Performance Computing (HPC) Basic Profile (HPC-BP) Simple API for Grid Applications (SAGA) Data Access and Integration set of specifications (WS-DAI-*) Web Services Resource Framework (WSRF)

Making location count Copyright © 2010, Open Geospatial Consortium OWS-6 WPS grid processing OGF enabled WPS - implementation –WPS to benefit from and integrate with distributed computing resources and technologies –Two potential ways to make use of OGF specifications, concepts and their implementations where identified encapsulating other resources integration alongside other services

Making location count Copyright © 2010, Open Geospatial Consortium Grid-enabled WPS implementations Airport Scenario –Trajectory Service (WPS JSDL Profile) Science and Technology Facilities Council (STFC), UK –Plume Rendering Service (WPS JSDL Profile) Institute for Geoinformatics (IfGI), University of Münster, Germany

Making location count Copyright © 2010, Open Geospatial Consortium Grid-enabled WPS implementations Debris Flow Scenario –Rainfall Data Interpolation (WPS HPC Basic Profile) Institute for Geoinformatics (IfGI), University of Münster, Geographic Information Systems (GIS) Research Center, Feng Chia University (FCU), Taiwan –Geophone Data Analysis (WPS HPC Basic Profile) Institute for Geoinformatics (IfGI), University of Münster, Germany Geographic Information Systems (GIS) Research Center, Feng Chia University (FCU), Taiwan

Making location count Copyright © 2010, Open Geospatial Consortium OGC online resources OWS-6 Geoprocessing Workflow Architecture Engineering Report r5 Bastian Schäffer This document covers geo-processing workflow best practices and methods in a SOA environment.

Making location count Copyright © 2010, Open Geospatial Consortium Debris Flow Monitoring System

Making location count Copyright © 2010, Open Geospatial Consortium Debris flow – scenario A debris flow is a fast moving mass of unconsolidated, saturated debris that looks like flowing concrete. Based on existing debris flow monitoring system (proprietary and monolithic software infrastructure) in Taiwan (running since 2003). Develop and implement open (OGC) standards-based service oriented architecture (SOA) for debris flow monitoring.

Making location count Copyright © 2010, Open Geospatial Consortium Debris flow - scenario

Making location count Copyright © 2010, Open Geospatial Consortium Debris flow – sensors and geo-processing Landslides and flooding are a threat on the mountainous island of Taiwan, due to typhoons and earthquakes. The Geographic Information Systems Research Center, Feng Chia University (GIS.FCU) in Taiwan has implemented OGC services for use in workflow, detecting and analyzing sensor data for emergency response. There is a demonstration showing the working network of debris flow sensors and examples of distributed services performing analysis and processing the sensor data.

Making location count Copyright © 2010, Open Geospatial Consortium Debris flow monitoring system

Making location count Copyright © 2010, Open Geospatial Consortium Debris flow monitoring system

Making location count Copyright © 2010, Open Geospatial Consortium Debris flow monitoring system

Making location count Copyright © 2010, Open Geospatial Consortium Debris flow monitoring system

Making location count Copyright © 2010, Open Geospatial Consortium Debris flow operational scenario

Grid-enabled WPS from IfGI

Making location count Copyright © 2010, Open Geospatial Consortium Encapsulation vs. integration Integration requires a WSRF binding for WPS

Making location count Copyright © 2010, Open Geospatial Consortium Encapsulation vs. Integration

Making location count Copyright © 2010, Open Geospatial Consortium From portal select desired theme and area of interest Wizard picks appropriate workflow for desired result Wizard Mozambique Disaster Management Information System (DMIS) Workflows Estimated rainfall accumulation and flood prediction model Flood Model Selected workflow automatically activates needed assets and models Baseline water level, flood waters and predicted flooding GEOSS AIP-2 flood prediction and response Led by NASA, Spot Image, Northrop Grumman, ERDAS

Making location count Copyright © 2010, Open Geospatial Consortium IP3 Client & Workflow engine IP3 Distributed Community Catalog/Mediator WCS - TOther Non-OGC ServicesGBIF Non-OGC Services OGC WPS Access to Model CSW GEOSS Portal WCSWFS req resp req resp req resp req resp req resp req resp Broker GEOSS AIP-2 biodiversity & climate change Led by CNR, Univ of Colorado, GBIF Research Scientist

Making location count Copyright © 2010, Open Geospatial Consortium SWE and geo-processing workflow 37 SOS ! SAS Sensor Net SAS Mission Control Center Access & Processing Node WFS WCSCSW WPS CSW Register Measurement Types SPS CSW SOS

Making location count Copyright © 2010, Open Geospatial Consortium The power of workflow Workflows can be formally described as programmes that invoke Web services that invoke other Web services that invoke other Web services. Analytical models, decision support tools and other complex resources can quickly deliver results in this way and optionally, provide documentation of the steps. There is an OGC Workflow Domain Working Group: this is a forum for describing, discussing and providing best practice guidance related to geospatial workflows using existing IT workflow standards.

Making location count Copyright © 2010, Open Geospatial Consortium Contact me

Thanks for your attention