Select, Overlay, Explore; Multidimensional data Maintain Distributed Data; Heterogeneous coding, access Connect providers to users; Homogenize data access Voyager Data Federation Services Built and Used by a Virtual Community Layered Map Time Chart ProvidersUsers Vector GIS Data XDim Data SQL Table OLAP Web Images Voyager Federation Web Services Publish, Find, Bind Data & Tool Catalog Uniform Access/Retrieval Scatter Chart

The Dvoy Project DVOY is Federated Information System for heterogeneous, multidimensional datasets Voyager is a generic graphic browser for the federated DVOY data. The initial Dvoy infrastructure is being developed at CAPITA, with NSF supportDvoy Further services for data access, processing and viewing are expected from the community The project evolution is to ride 'web services wave‘ of the Internet CAPITA projects which use DVOY: –NSF ITRWorkgroup Collaboration Tool:Aug Aug 2004 –EPA Web-based Visibility:Aug Apr 2003 –NOAAASOS Visibility:Sep Sep 2002 –MARAMAChemical Trajectory Tool:Aug July 2003 –EPA OAQPS Global Transport Analysis:Nov 2002 – Oct 2003 –NSF DigiGov Fire and Smoke Network :May 2003 – Apr 2006 Pending –NASA ESE Satellite Appl. to PM Management :June 2003 – May 2008 Pending In-kind support by organizations participating in DVOY-based federated data sharing Collaborators/Partners: CIRA (Schichtel), NRL(Westphal), NASA (Goddard)…many data sources. CAPITA cast: R. Husar, S. Falke, K. Hoijarvi, J. Colson, R. Zager)

Multi-Dimensional Data Model Data can be distributed over 1,2, …n dimensions 1 Dimensional e.g. Time dimension i j k j i Data Granule i 1 Dimensional e.g. Location & Time 1 Dimensional e.g. Location, Time & Parameter View 1 Data Space View 2 Views are orthogonal slices through multidimensional data cubes Spatial and temporal slices through the data are most common

Render Service Chaining in Spatio-Temporal Data Browser Spatial Slice Find/Bind Data Data Cube Time Slice Time Portrayal Spatial PortrayalSpatial Overlay Time Overlay OGC-Compliant GIS Services Time-Series Services PortrayOverlay Homogenizer Catalog Wrapper Mediator Client Browser Cursor/Controller Maintain Data Vector GIS Data XDim Data SQL Table OLAP Satellite Images Data Sources

Overlay of multiple Datasets Each DataCube may have 0-n dimensions Each dimension is assigned a view 3 D DataCube 2 D DataCube DataView 3 Layer 2 Layer 1 DataView 1 DataView 2 In a view, the number of layers is the number of datasets If a DataCube does not have a data for a view, a Null Layer is assigned Null Layer

Overlay of multiple Datasets Each DataCube may have 0-n dimensions Each dimension is assigned a view DataView 3 DataView 1 DataView 2 In a view, the number of layers is the number of datasets If a DataCube does not have a data for a view, a Null Layer is assigned 3 D DataCube Data Access Connections Data Render Connections

Federated Information System Data producers maintain their own workspace and resources (data, reports, comments). However, part of the resources are shared through a Federated Information System. Web-based integration of the shared resources can be across several dimensions: Planned data sharing federations: Open GIS Consortium (GIS data layers) NASA SEEDS network (Satellite data) NSF Digital Government EPA’s National Env. Info Exch. Network. VIEWSRPO NASA NAAPS RPO Federated Data System Data, Tools, Methods SharedPrivate RPO Other Federations Applications PM Policy Regulation Mitigation

Idaho Fires, 2000: Overlaying Fire Locations, PM2.5, SewWiFS PM2.5 data from the IMPROVE and AIRS database at Colorado State (VIEWS) CIRA ColoState-VIEWS European Space Agency Generic Map Server NASA SeaWiFS Project CAPITA SeaWiFS AOT FEDERATED DATA SOURCES: Fire locations from European Space Agency (Project IONIA) Reflectance image from NASA SeaWiFS Project

Idaho Fires Example (August 2000) ASTR Fire Locations Processed SeaWiFS image showing aerosol optical depth Derived aerosol optical depth provides a focused look at smoke dispersion The SeaWiFS reflectance image is hidden to expose the aerosol optical depth image ‘below’ it

Web Services Components and Actions Service providers publish services to a service broker. Service users find the needed service and get access key from a service broker With the access key, users bind to the service provider The result is a dynamic binding mechanism between the service users and providers Service Broker Service Provider Publish Find Bind Service User Components:Provider – User – Broker Actions: Publish – Find - Bind

Web Publish HTTP, FTP Data Access though a Web Service Adapter Service Cataloger Service Consumer Ordinary web content can be delivered as a Web Service through a Proxy Server. The Wrapper Service converts HTTP/FTP service to XML Web service The Wrapper Service is published as a web service to the Broker The User finds the service from the broker and accesses the Wrapper to use the service Service Wrapper HTTP/FTP Service User Chain Find UDDI, WSDL Publish UDDI, WSDL Access SOAP, XML

Interoperability through a Layered Protocol Stack Web Services are implemented on a layered stack of technologies and standards The lower layers enable binding and exchange of messages; higher levels enable interoperability Applications are formed dynamically from distributed components through publish-find-bind mechanisms TCP/IP, HTTP, FTP ASCII, XML, etc. HTML, XML OGC -GML OGC Coverage, CoordTransfom, WMS HTTP, SOAP WSDL UDDI OGC Catalog WSFL, XLANG Standards Interoperability Comm. Protocols Data Encoding Data Schema Data Binding Web Service Service Integr. Service Discovery Service Descript. Connectivity

Web Application: Chained Web Services A Web Service Provider may also be a User of other services Multiple web services can be chained into an interactive workflow system The result is an agile application that can be created ‘just in time’ by the user for a specific need Service Broker Service Provider/User Publish Find Bind Service User Chain Service Provider Bind Chain

Architecture of Dvoy Federated Information System The main software components of Dvoy are wrappers, which encapsulate sources and remove technical heterogeneity, and mediators, which resolve the logical heterogeneity (e.g. Busse et. al., 1999). Wrapper classes are available for geo-spatial (incl. satellite) images, SQL servers, text files,etc. The mediator classes are implemented as web services for uniform data access to n-dimensional data.

4 D Geo-Environmental Data Cube (X, Y, Z, T) Environmental data represent measurements in the physical world which has space (X, Y, Z) and time (T) as its dimensions. The inherent dimensions for geo-environmental data are: Longitude X, Latitude Y, Elevation Z and Time T.

Typical Slices through XYZT Data Cube Possible Cross-sections through the 4 D Data space - data point..Temperature (x i, y i, z i, t i ) - image Temperature (x range, y range, z range, t range ) XY MAP: Z,T fixed Vertical Profile:XYT fixed Time Chart: X,Y,Z fixed Vertical Cross sect: YT fixedVertical Cross sect: XT fixed Vertical Profile Trend: X,Y fixed

Features of the DVOY XML Web Service Architecture Interoperability: Platform and language independence; based on Web Services (XML,SVG) Legacy Support: Encapsulating existing data and exposing them as Web Services. (Access to standard HTTP/FTP servers) Just-in-time integration: Discovery, access to and ad-hoc chaining of services. (Future: Agile application building) DVOY draws on three basic web services: DataCatalogWrapping service is for data registration, finding and wrapping information DataAccess service provides uniform access to heterogeneous, distributed, multidim. data DataPortrayal service prepares input data in form suitable for rendering

Data Catalog All the data in the system are to be distributed on the Web and maintained by their custodians The purpose of the catalog is to help finding and and accessing the data Catalog would be limited to data that can be accessed/merged in DVOY

Catalog of Multidimensional Datasets Designed to Publish, Find(Select), Bind(Access) and Render distributed datasets Publishing is though an open web interface for user/broker registration of datasets Finding a dataset is aided by a metadata on Provider and Dataset Binding (data access) information is contained in the Dimensional Tables Rendering parameters are also contained in the Dimensional Tables

Data Catalog

Geo-referenced Map Image: Image Indexed Variable Time, Var GeoRect, Var Sensor SatGeoImg CapImgIDUniversal ID ProviderURLURL SourceURLURL ImageURLURL ProviderAbbrevNASA GSFC ImageDescriptionText SensorEnumerated DateTime LatMin-65 LatMax 65 LonMin- 180 LonMax180 ImageWidthFor size verification ImageHeight MarginLeftFor trimming MarginTop MarginRight MarginBottom SQL Table … see table left NO DIMENSION TABLES XML Example: SeaWiFS

Geo-referenced Map Image: Time Indexed Regular Time Series; Fixed Georect; Image Trim < time_dimension handler_assembly="CAPITA.data.index.impl" handler_class="CAPITA.data.index.impl.datetime_interval" start_time=" " end_time="yesterday"> sample_periodicity _unit = "day" sample_periodicity_mult = “1" TOMSFact TimeIndex Int, Calc from date DailyImage Img, Parsed from date LatMin Fixed, -65 LatMax Fixed, 65 LonMin Fixed, LonMax Fixed, 180 TOMSTime TimeIndex DateTime Virtual Tables (No physical StarTables)XML Example: TOMS ftp://jwocky.gsfc.nasa.gov/pub/eptoms/images/aerosol/y2002/ea gif

Astronaut Photos: Image Indexed Variable Time, Image Center, Platform SatImg NASAImageIDUniversal ID ProviderURL SourceURLhttp://eol.jsc.nasa.gov/script s/sseop/photo.pl?mission=IS S004&roll=E&frame=11079 ImageURLhttp://eol.jsc.nasa.gov/sseop/ images/ESC/small/ISS004/I SS004-E JPG ProviderAbbrevNASA JSC ImageDescriptionText PlatformISS004 GeoRegionGUATEMALA FeaturesSMOKE, GULF OF HONDURAS DateTime Lat17.6 Lat-90.2 SQL Table <data_source service_url=" /> … see table left XML Example: ISS004

Atmospheric Photos: Image Indexed Variable Time, Image Center, Platform AtmoImg ImageIDUniversal ID ProviderURLhttp://capita.wustl.edu/capita /people/RHusar/rhusar.html SourceURLhttp://capita.wustl.edu/capita /people/RHusar/Pic/9812San FranciscoAGU/9812SanFra nciscoAGU.htm ImageURLhttp://capita.wustl.edu/capita /people/RHusar/Pic/9812San FranciscoAGU/Dcp01000E nh.jpg Provider AbbrevR Husar ImageDescriptionText PlatformAirplane GeoRegionNEVADA FeaturesDUST PLUME DateTime Lat36.75 Lat SQL Table <data_source service_url=" /> … see table left XML Example: Dust Plume Photo

DVOY State Maintenance This DVoyState data structure contains the state variables, which drive DVoy during browsing It is created by the DataCatalogWarpping service using the Catalog and Wrapping information User actions on Controllers modify the DVoyState structure directly but do not execute changes Actions are executed by managers (e.g. ConMan for connections) based on changes in DVoyState DVoyState resides on the client web page, hence it is unique for each Dvoy session The session state can be saved as a web page and can be recreated at load time Workbooks can be created from multiple pages of DVoyState (not yet implemented)

Dvoy Catalog Interaction: Find -> Bind WSGetDatasetList DatasetList.xmlDVoyState.xml WSGetDatasetInfo DatasetInfo.xml DatasetSelector DVoy Catalog Database On LoadCatalogPage On AddLayer Init DVoyStateAdd layer/dataset_abbr to DVoyStateAdd Layer Description to DVoyState DVoyState.xml

Dvoy Catalog Interaction: Find -> Bind DataSelectService – performs the FIND web service operation Queries the catalog for the list of datasets through GetDatasetList service Passes the returned DatasetList to a controller (listbox) where the user selects a dataset DataSelectService places selected DataCode into DVoyState.xml DataInfoService – performs the BIND web service operation Accepts data generic data request: GetAccessInfo (DatasetCode, Dimension, Param, Location, Time, ViewConstr) retrieves or generates access instructions for the query, places them into DVoyState.xml DataInfoService exposes the DataSelectService exposes the selected DataCode DatasetList DataInfoService AccessInfo DataSelectService DVoy Catalog Database On AddLayer DVoyState.xml GetDatasetList Browsing Environment CursorCodes DatasetCode Find Bind GetAccessInfo User User Selects New Data layer

DVOY Interfaces Data Input Data input Data Output - Browser The DVOY interface is composed of data viewers and controllers, all displayed on a webpage The web services and the preparation of the webpage interface is through.NET(Microsoft) The graphic data display on the webpage uses an SVG plugin (Adobe) The DVOY controls are linked to the SVG plugin and the.NET through client-side JavaScript Data Output – Web Service –The DVOY outputs are XML formatted datasets suitable for chaining with processing or rendering services

NRL Forecast Model for Dust, Smoke and Sulfate METAR Surface Haze Real-time model and surface observations are compared spatially and temporally Dust Sulfate Smoke METAR Haze Time Selector

Quebec Fires, July 6, 2002 SeaWiFS, METAR and TOMS Index superimposed SeaWiFS satellite and METAR surface haze shown in the Voyager distributed data browser Satellite data are fetched from NASA GSFC; surface data from NWS/CAPITA servers

Trans-Atlantic Transport of Quebec Smoke July 11: Smoke approaching Europe July 10: Quebec smoke over Mid- Atlantic SeaWiFS Reflectance TOMS Absorbing Aerosol SeaWiFS Reflectance TOMS Absorbing Aerosol Spain E. US

Browsing of Distributed Data from HTTP/FTP Servers Land Reflectance from SeaWiFS Project, NASA GSFC AVHRR Oceanic Aerosol CAPITA, WashU, STL Fire Pixels, Jan 1997, ESA Ionia Project

Voyager Spatio-Temporal Data Browser

NSF-NOAA-EPA/EMAP (NASA)? Project: Real-Time Aerosol Watch System Real-Time Virtual PM Monitoring Dashboard. A web-page for one-stop access to pre-set views of current PM monitoring data including surface PM, satellite, weather and model data. Virtual Workgroup Website. An interactive website which facilitates the active participation of diverse members in the interpretation, discussion, summary and assessment of the on-line PM monitoring data. Air Quality Managers Console. Helps PM managers make decisions during major aerosol events; delivers a subset of the PM data relevant to the AQ managers, including summary reports prepared by the Virtual workgroups.

wind directionback trajectoriestemperatureNAAPS modelPM/Bext time seriesBext contoursPM2.5 contoursweb cam Real-time PM Monitoring Dashboard Example Views – Selected from Dozens of spatial, temporal, height cross-sections satellite image