Sensor Web Enablement (SWE) and SensorML January 2008 Mike Botts mike.botts@uah.edu Principal Research Scientist University of Alabama in Huntsville

OpenGeospatial Consortium (OGC)

Open Geospatial Consortium The Open Geospatial Consortium, Inc (OGC) is an international industry consortium of 334+ companies, government agencies and universities participating in a consensus process to develop publicly available interface specifications and encodings. Open Standards development by consensus process Interoperability Programs provide end-to-end implementation and testing before spec approval Standard encodings (e.g. GML, SensorML, O&M, etc.) Geography Markup Language (GML) – Version 3.2 Style Layer Description language (SLD) SensorML Observations and Measurement (O&M) Standard Web Service interfaces; e.g.: Web Map Service (WMS) Web Feature Service (WFS) Web Coverage Service (WCS) Catalog Service Open Location Services – used by communications and navigation industry Sensor Web Enablement Services (SOS, SAS, SPS)

Sensor Web Enablement

Basic Desires Quickly discover sensors and sensor data (secure or public) that can meet my needs – location, observables, quality, ability to task Obtain sensor information in a standard encoding that is understandable by me and my software Readily access sensor observations in a common manner, and in a form specific to my needs Task sensors, when possible, to meet my specific needs Subscribe to and receive alerts when a sensor measures a particular phenomenon

Sensor Web Enablement Framework Heterogeneous sensor network In-Situ monitors Bio/Chem/Rad Detectors Surveillance Airborne Satellite sparse disparate mobile/in-situ extensible Decision Support Tools vendor neutral extensive - flexible adaptable Sensor Web Enablement discovery access tasking alert notification web services and encodings based on Open Standards (OGC, ISO, OASIS, IEEE) Models and Simulations nested national, regional, urban adaptable data assimilation M. Botts -2004

Background -1- 1999 - 2000 2001 2002 2003-2004 OGC Web Services Testbed 1.2: Sponsors: EPA, General Dynamics, NASA, NIMA Specs: SOS, O&M, SensorML, SPS, WNS Demo: Terrorist, Hazardous Spill and Tornado Sensors: weather stations, wind profiler, video, UAV, stream gauges Specs advanced through independent R&D efforts in Germany, Australia, Canada and US Sensor Web Work Group established Specs: SOS, O&M, SensorML, SPS, WNS, SAS Sensors: wide variety OGC Web Services Testbed 1.1: Sponsors: EPA, NASA, NIMA Specs: SOS, O&M, SensorML Demo: NYC Terrorist Sensors: weather stations, water quality SensorML initiated at University of Alabama in Huntsville: NASA AIST funding 1999 - 2000 2001 2002 2003-2004

OGC Web Services Testbed 4.0: Background -2- OGC Web Services Testbed 3.0: Sponsors: NGA, ORNL, LMCO, BAE Specs: SOS, O&M, SensorML, SPS, TransducerML Demo: Forest Fire in Western US Sensors: weather stations, wind profiler, video, UAV, satellite SAS Interoperabilty Experiment OGC Web Services Testbed 4.0: Sponsors: NGA, NASA, ORNL, LMCO Specs: SOS, O&M, SensorML, SPS, TransducerML, SAS Demo: Emergency Hospital Sensors: weather stations, wind profiler, video, UAV, satellite OGC Web Services Testbed 5.1 Sponsors: NGA, NASA, Specs: SOS,SensorML,TML Demo: Streaming JPIP of Georeferenceable Imagery; Geoporocessing Workflow Sensors: Satellite and airborne imagery SWE Specifications toward approval: SensorML – V0.0 TransducerML – V0.0 SOS – V0.0 SPS – V0.0 O&M – Best Practices SAS – Best Practices 2005 2006 2007

SWE Specifications Information Models and Schema Web Services Sensor Model Language (SensorML) for In-situ and Remote Sensors - Core models and schema for observation processes: support for sensor components, georegistration, response models, post measurement processing Observations and Measurements (O&M) – Core models and schema for observations TransducerML – adds system integration and multiplex streaming clusters of observations Web Services Sensor Observation Service - Access Observations for a sensor or sensor constellation, and optionally, the associated sensor and platform data Sensor Alert Service – Subscribe to alerts based upon sensor observations Sensor Planning Service – Request collection feasibility and task sensor system for desired observations Web Notification Service –Manage message dialogue between client and Web service(s) for long duration (asynchronous) processes Sensor Registries – Discover sensors and sensor observations

Status Current specs are in various stages SensorML (and SWE Common) – Version 1.0.1 TransducerML – Version 1.0 Observations & Measurement – Version 1.0 WNS – Request for Comments SOS – Version 1.0 SPS – Version 1.0 SAS – Request for Comments Approved SWE standards can be downloaded: Specification Documents: http://www.opengeospatial.org/standards Specification Schema: http://schemas.opengis.net/

Known Open Source Software for SWE 52 North / University of Muenster Full suite of SWE services (SOS, SPS, SAS, WNS) University of Alabama in Huntsville SWE Common parser/writer, SensorML parser, process chain executor and process model library editors for SensorML/O&M instances and profiles Space Time Toolkit SWE client SOS/WCS services SWE portrayal service (initially KML) Texas A&M / Marine Metadata Initiative Non ebRIM registry based on ontology light weight clients, several services MapServer / GDAL SWE services incorporated into MapServer NASA GSFC / GeoBlinky Several components used with the EO1 SAT activities Northrop Grumman Several components used within the PulseNet activity SAIC Ongoing development of several Open-Source SWE components under MASINT funding There is an initiative to begin to look at joint development and management of Open Source SWE software

Example Known External Activities using SWE Community Sensor Models (NGA) – SensorML encoding of the CSM; CSM likely to be the ISO19130 standard Multi-Int Metadata Standards (DIA) – Committed to SensorML and SWE as direction OGC OWS5.1 Georeferenceable Imagery (NGA/NASA) – will be demonstrating use of SensorML within JPEG2000 and JPIP for support of geolocation of streaming imagery Oak Ridge National Labs SensorNet – funded project will be adding SensorML support in SensorNet nodes for threat monitoring Northrop Grumman IRAD (NGC TASC) – demonstrated end-to-end application of SensorML/SWE for legacy surveillance sensors in field Empire Challenge (NGA - SAIC) – planning to test SWE for sensor observation processing and integration in 2008 testbed European Space Agency – developing SensorML profiles for supporting sensor discovery and processing within the European satellite community Canadian GeoConnections Projects – used SensorML in water monitoring network Hurricane Missions (NASA MSFC) – using SensorML for geolocation and processing of satellite and airborne sensors Sensors Anywhere (SAny) – intending to use SensorML/SWE Common within large European sensor project NASA ESTO – funded 30 3-year projects on Sensor Webs; 5 SBIR topics with SensorML and Sensor Web called out

SensorML

What is SensorML? XML encoding for describing sensor processes Including sensor tasking, measurement, and post-processing of observations Detectors, actuators, sensors, etc. are modeled as processes Open Standard – Approved by Open Geospatial Consortium in 2007 Supported by Open Source software (COTS development starting) Not just a metadata language enables on-demand execution of algorithms Describes Sensor Systems Processing algorithms and workflows

Why is SensorML Important? Importance: Discovery of sensors and processes / plug-n-play sensors – SensorML is the means by which sensors and processes make themselves and their capabilities known; describes inputs, outputs and taskable parameters Observation lineage – SensorML provides history of measurement and processing of observations; supports quality knowledge of observations On-demand processing – SensorML supports on-demand derivation of higher-level information (e.g. geolocation or products) without a priori knowledge of the sensor system Intelligent, autonomous sensor network – SensorML enables the development of taskable, adaptable sensor networks, and enables higher-level problem solving anticipated from the Semantic Web

SensorML Processes Physical Processes Non-Physical Processes Processes where physical location or physical interface of the process is not important (e.g. a fast-Fourier process) Processes where physical location or physical interface of the process is important (e.g. a sensor system) Atomic Processes Composite Processes Processes that are considered Indivisible either by design or necessity Processes that are composed of other processes connected in some logical manner

Example Atomic Processes Transducers (detectors, actuators, samplers, etc.) Spatial transforms (static and dynamic) Vector, matrix, quaternion operators “Sensor models” scanners, frame cameras, SAR polynomial models (e.g. RPC, RSM) tie point model Orbital models Geospatial transformations (Map projection, datum, coordinate system) Digital Signal Processing / image processing modules Decimators, interpolators, synchronizers, etc. Data readers, writers, and access services Derivable Information (e.g. wind chill) Human analysts

Example Composite Processes Sensor Systems, Platforms Observation lineage from tasking to measurement to processing to analysis Executable on-demand process chains: geolocation and orthorectification algorithms for higher-level products e.g. fire recognition, flood water classification, etc. Image processing, digital signal processing Uploadable command instructions or executable processes

Status of SensorML and SWE Common SensorML history Influenced by interoperability challenges for satellite sensors at NASA Started at UAH in 1998 under NASA AIST funding; brought into OGC in 2000 Approved as Public Discussion Paper (2002) Approved as Recommended Paper (2004) OGC 05-086 approved as Best Practices Document in Bonn (Nov 2005) OGC 05-086r3 approved as Version 0.0 Technical Specification in July 2006 OGC 07-000 approved as Technical Specification Version 1.0 on June 23, 2007 Current: document (OGC 07-000) Approved Version 1.0 of SensorML and SWE Common data types Official document available at OGC ( http://www.opengeospatial.com ) Official Reference Schema resides online at http://schemas.opengis.net/ Doc and schema also available at http://vast.uah.edu/SensorML

Where has SensorML been demonstrated and tested ?

Known Demonstrations and Testbeds for SensorML Previous OGC OWS Testbeds OWS 1.1 (2001) – description and access to in-situ sensors OWS 1.2 (2002) – discovery, access, and georeferencing of remote sensors; fusion with in-situ sensors OWS 3.0 (2005) – discovery, on-demand processing of radar, satellite, UAV, weather station, and plume model observations OWS 4.0 (2006) – discovery, on-demand processing of CBRNE, plume model, and weather sensors NASA SMART ODM (2006-present) – have used SensorML to georeference satellite data and to automatic determination of coincidence between sensor and numerical model data Northrop Grumman IRAD (PulseNet 2006-2007) – demonstrated end-to-end application of SensorML/SWE for legacy surveillance sensors in field Empire Challenge 2007 (NGA) – SensorML used for discovery and data access of disparate sensor sources Canadian GeoConnections Projects (2005) – used SensorML in water monitoring network (discovery and data access) NASA Hurricane Missions (2006-present) – using SensorML for geolocation and processing of satellite and airborne sensors NASA SensorML Project (2006-present) – incorporation and demonstration of SensorML execution engine into Space Time Toolkit and SWE services

Previous OGC OWS Testbeds: SensorML-Enabled Discovery and Georeferencing Weather LaPlata Tornado UAV for Fire Detection Radiation plumes and weather

A Few Known Ongoing Activities using SensorML Community Sensor Models (NGA) – SensorML encoding of the CSM; CSM likely to be the ISO19130 standard Multi-INT Metadata Standards (DIA and DISA) – Committed to SensorML and SWE as direction OGC OWS5.1 Georeferenceable Imagery (NGA/NASA) – will be demonstrating use of SensorML within JPEG2000 and JPIP for support of georeferencing of streaming imagery Oak Ridge National Labs SensorNet – funded project will be adding SensorML support in SensorNet nodes for threat monitoring (including georeferenced streaming video) Empire Challenge 2008 (NGA) – planning to test SensorML for sensor observation processing and fusion in 2008 testbed European Space Agency – developing SensorML profiles for supporting sensor discovery and processing within the European satellite community Hurricane Missions (NASA MSFC) – working toward using SensorML for geolocation and processing of satellite and airborne sensors during real-time missions Sensors Anywhere (SAny) – intending to use SensorML/SWE Common within large European sensor project NASA– funded 30 3-year projects developing capabilities for SensorML and Sensor Webs; Also recently announce call for SBIR proposals with SensorML and Sensor Web topics identified

PulseNet: SensorML-Enabled Discovery, Data Access, and Tasking Credit: Northrop Grumman PulseNet Project

NASA Projects: SensorML-Enabled On-demand Processing (e. g NASA Projects: SensorML-Enabled On-demand Processing (e.g. georeferencing and product algorithms) AMSR-E SSM/I TMI & MODIS footprints MAS TMI Geolocation of satellite and airborne sensors using SensorML Cloudsat LIS

SensorML – Sensor Systems System - Aircraft IR radiation Sensor 1 Scanner Digital Numbers Sensor 2 IMU Pitch, Roll, Yaw Tuples Attitude We also describe sensor systems = collection of sensors with I/O = chain of interconnected atomic sensors A system also defines connections between components which is even more important on the next slide Sensor 3 GPS Lat, Lon, Alt Tuples Location Mike Botts, Alexandre Robin, Tony Cook - 2005

AIRDAS UAV Geolocation Process Chain Demo AIRDAS data stream geolocated using SensorML-defined process chain (software has no a priori knowledge of sensor system) AIRDAS data stream (consisting of navigation data and 4-band thermal-IR scan-line data)

SensorML for Discovery

SensorML provides metadata suitable for discovery of sensors and processes Find all remote sensor systems measuring in the visible spectral range with ground resolution less than 20m.

Discovery Based on SensorML Credit: Northrop Grumman PulseNet Project

Specific Discovery Needs Unique resource ids used throughout SWE; sensor centric example: Find sensors that can do what I need (returns id=“urn:ogc:id:sensor:123”) Get me a full description of this sensor urn:ogc:id:sensor:123 Now, find a service (SPS) that lets me task this sensor urn:ogc:id:sensor:123 Find all services (SOS) where I can get observations from this sensor urn:ogc:id:sensor:123 Find all processes that can be applied to this sensor output to generate the information I require Catalog profiles for each need: SPS, SOS, SAS services sensors and processes observations terms (either through dictionaries or ontologies)

Need for Term Definitions used in SensorML and SWE Observable properties / phenomena / deriveable properties (“urn:ogc:def:property:*) temperature, radiance, species , exceedingOfThreshold, earthquake, etc. rotation angles, spectral curve, histogram, etc. Capabilities, Characteristics, Interfaces, etc. (“urn:ogc:def:property:*”) Width, height, material composition, etc. Ground resolution, dynamic range, peak wavelength, etc. RS-232, USB-2, bitSize, baud rate, base64, etc. Sensor and process terms (“urn:ogc:def:property:*”) IFOV, focal length, slant angle, etc. Polynomial coefficients, matrix, image, etc. Identifiers and classifiers (“urn:ogc:def:identifierType:*; urn:ogc:def:identifier:*” ) Identifiers – longName, shortName, model number, serial number, wingID, missionID, etc. Classifiers – sensorType, intendedApplication, processType, etc. Role types (“urn:ogc:def:role:*”) Expert, manufacturer, integrator, etc. Specification document, productImage, algorithm, etc. Sensor and process events (“urn:ogc:def:classifier:eventType:*”) Deployment, decommissioning, calibration, etc.

Help, Help, Help We need authoritative bodies with access to subject-matter-experts (SME) to step forward to establish resolvable term dictionaries for sensors, processes, and observations Potential authoritative bodies IC community – GIG, MASINT WG, Multi-INT Interoperability Lab ?? Civilian satellite community – Committee for Earth Observation Satellites (CEOS) Others - ??? Way forward Create namespace for terms Develop web interface for submitting term (Wikipedia perhaps, XML-based?) Term Definition References Relationship (?) – or allow separate ontologies to provide this Level of authorization Set up web services for resolving and getting filtered list of terms Set up authentication process and authentication levels (e.g. submitted, under review, approved, rejected) Accepting SensorML and SWE without creating authorized terms won’t accept interoperability

Where and how SensorML can be used

Supports description of Lineage for an Observation SensorML Observation Within an Observation, SensorML can describe how that Observation came to be using the “procedure” property

On-demand processing of sensor data SensorML Observation SensorML processes can be executed on-demand to generate Observations from low-level sensor data (without a priori knowledge of sensor system)

On-demand processing of higher-level products SensorML Observation Observation SensorML processes can be executed on-demand to generate higher-level Observations from low-level Observations (e.g. discoverable georeferencing algorithms or classification algorithms)

On-demand Geolocation using SensorML AMSR-E SSM/I TMI & MODIS footprints MAS TMI Geolocation of satellite and airborne sensors using SensorML Cloudsat LIS

Clients can discover, download, and execute SensorML process chains SensorML-enabled Client (e.g. STT) SLD SensorML OpenGL SOS Stylers For example, Space Time Toolkit is designed around a SensorML front-end and a Styler back-end that renders graphics to the screen

Incorporation of SensorML into Space Time Toolkit Space Time Toolkit being retooled to be SensorML process chain executor + stylers

Space Time Toolkit Sample Applications -2-

SensorML-Enabled Web Services

SensorML can support generation of Observations within a Sensor Observation Service (SOS) SOS Web Service SensorML Observation request For example, SensorML has been used to support on-demand generation of nadir tracks and footprints for satellite and airborne sensors within SOS web services

Incorporation of SensorML into Web Services SensorML process chains have been used to drive on-demand data within services (e.g. satellite nadir tracks, sensor footprints, coincident search output)

SensorML can support tasking of sensors within a Sensor Planning Service (SPS) SPS Web Service SensorML request For example, SensorML will be used to support tasking of video cam (pan, tilt, zoom) based on location of target (lat, lon, alt)

SPS control of Web Cam

SensorML for Portrayal

SWE Visualization Clients can render graphics to screen SensorML-enabled Client (e.g. STT) SLD SensorML OpenGL SOS Stylers

SWE Portrayal Service can “render” to various graphics standards SLD SensorML KML Collada SOS Google Earth Client Stylers For example, a SWE portrayal service can utilize a SensorML front-end and a Styler back-end to generate graphics content (e.g. KML or Collada)

SensorML to Google Earth (KML – Collada) AMSR-E SSM/I MAS TMI LIS

SensorML Support Activities

Current Tool Development and Support for SensorML SensorML Forum – mail list for SensorML discussion (250+ active members from various backgrounds) http://mail.opengeospatial.org/mailman/listinfo/sensorml Open Source SensorML Process Execution Engine – Along with open-source process model library, provides execution environment for SensorML described algorithms Open Source SensorML editor and process chain development client – on-going development of tools to allow human-friendly editors for SensorML descriptions SensorML-enabled decision support client – Open source Space Time Toolkit is SensorML-enabled and will be available to discover, access, task, and process sensor observations; use as is or as template for COTS development SensorML white papers and tutorials – being written and released on an array of SensorML topics Describing a Simple Sensor System Creating New Process Models;

SensorML Process Editors Currently, SensorML documents are edited in XML (left), but will soon be edited using human friendly view (below) Currently, we diagram the process (right top) and then type the XML version; soon the XML will be generated from the diagram itself (right bottom)

Simple SensorML Forms for the Mass Market User fills out simple form with manufacturer name and model number, as well as other info. Then detailed SensorML generated.

How can you help? Investigate SensorML for you needs Can it meet some of your needs? What would be your main initial focus (discovery, lineage, processing, display)? SensorML Forum – participate in the forum; ask/answer questions Term Dictionaries – help develop online dictionary of sensor terms used in your community Sensor and process profile – develop SensorML profiles for your sensors and processes Test and demonstrate – Test SensorML applications for your environment Demonstrate successes Feedback failures, suggestions, and additional needs Participate in the OGC process Join and attend Technical Committee meeting (particularly SWE WG) Sponsor Interoperability Projects in OGC

Relevant Links Open Geospatial Consortium Standard Documents http://www.opengeospatial.org/standards OGC Approved Schema http://schemas.opengis.net/ Sensor Web Enablement Working Group http://www.opengeospatial.org/projects/groups/sensorweb SensorML information http://vast.uah.edu/SensorML SensorML Public Forum http://mail.opengeospatial.org/mailman/listinfo/sensorml