1. VOYAGER Data Explorer: Architecture and Technologies See also the the Voyager Developer Website and early ApplicationsDeveloper WebsiteApplications Layered Map Time Chart ProvidersUsers Vector GIS Data XDim Data SQL Tables Web Images Voyager Web Services Publish, Find, Bind Data & Tool Catalog Uniform Access Scatter Chart S u p p o r tCoord./Cooperation T e c h n o l o g i e s

2. Select, Overlay, Explore; Multidimensional data Maintain Distributed Data; Heterogeneous coding, access Connect providers to users; Homogenize data access Voyager Spatio-Temporal Data Explorer Built and Used by a Virtual Community Layered Map Time Chart ProvidersUsers Vector GIS Data XDim Data SQL Table OLAP Web Images Voyager Web Services Publish, Find, Bind Data & Tool Catalog Uniform Access/Retrieval Scatter Chart

3. The Dvoy Project DVOY is a graphic browser for distributed, heterogeneous, multidimensional datasets The initial Dvoy infrastructure was developed at CAPITA, with NSF supportDvoy Further services for data access, processing and viewing are expected from the community The project evolution is to ride 'web services wave‘ of the Internet CAPITA Support: –NSF ITRWorkgroup Collab. Tool:Aug 2001 - Aug 2004 –EPA Web-based Visibility:Aug 2001 - Apr 2003 –NOAAASOS Visibility:Aug 2001 - May 2002 –MARAMAChemical Trajectory Tool:Aug 2002 - July 2003 –EPA OAQPS Global Transport Analysis:Nov 2002 – Oct 2003 In-kind support by organizations participating in DVOY-based data sharing

4. Multi-Dimensional Data Model Data can be distributed over 1,2, …n dimensions 1 Dimensional e.g. Time dimension i j k j i Data Granule i 1 Dimensional e.g. Location & Time 1 Dimensional e.g. Location, Time & Parameter View 1 Data Space View 2 Views are orthogonal slices through multidimensional data cubes Spatial and temporal slices through the data are most common

5. Render Service Chaining in Spatio-Temporal Data Browser Spatial Slice Find/Bind Data Data Cube Time Slice Time Portrayal Spatial PortrayalSpatial Overlay Time Overlay OGC-Compliant GIS Services Time-Series Services PortrayOverlay Homogenizer Catalog Wrapper Mediator Client Browser Cursor/Controller Maintain Data Vector GIS Data XDim Data SQL Table OLAP Satellite Images Data Sources

6. Overlay of multiple Datasets Each DataCube may have 0-n dimensions Each dimension is assigned a view 3 D DataCube 2 D DataCube DataView 3 Layer 2 Layer 1 DataView 1 DataView 2 In a view, the number of layers is the number of datasets If a DataCube does not have a data for a view, a Null Layer is assigned Null Layer

7. 4 D Geo-Environmental Data Cube (X, Y, Z, T) Environmental data represent measurements in the physical world which has space (X, Y, Z) and time (T) as its dimensions. The inherent dimensions for geo-environmental data are: Longitude X, Latitude Y, Elevation Z and Time T.

8. Typical Slices through XYZT Data Cube Possible Cross-sections through the 4 D Data space - data point..Temperature (x i, y i, z i, t i ) - image Temperature (x range, y range, z range, t range ) XY MAP: Z,T fixed Vertical Profile:XYT fixed Time Chart: X,Y,Z fixed Vertical Cross sect: YT fixedVertical Cross sect: XT fixed Vertical Profile Trend: X,Y fixed

9. Web Services

10. Web Services Components and Actions Service providers publish services to a service broker. Service users find the needed service and get access key from a service broker With the access key, users bind to the service provider The result is a dynamic binding mechanism between the service users and providers Service Broker Service Provider Publish Find Bind Service User Components:Provider – User – Broker Actions: Publish – Find - Bind

11. Interoperability through a Layered Protocol Stack Web Services are implemented on a layered stack of technologies and standards The lower layers enable binding and exchange of messages; higher levels enable interoperability Applications are formed dynamically from distributed components through publish-find-bind mechanisms TCP/IP, HTTP, FTP ASCII, XML, etc. HTML, XML OGC -GML OGC Coverage, CoordTransfom, WMS HTTP, SOAP WSDL UDDI OGC Catalog WSFL, XLANG Standards Interoperability Comm. Protocols Data Encoding Data Schema Data Binding Web Service Service Integr. Service Discovery Service Descript. Connectivity

12. Web Publish HTTP, FTP Data Access though a Web Service Adapter Service Cataloger Service Consumer Ordinary web content can be delivered as a Web Service through a Proxy Server. The Wrapper Service converts HTTP/FTP service to XML Web service The Wrapper Service is published as a web service to the Broker The User finds the service from the broker and accesses the Wrapper to use the service Service Wrapper HTTP/FTP Service User Chain Find UDDI, WSDL Publish UDDI, WSDL Access SOAP, XML

13. Web Application: Chained Web Services A Web Service Provider may also be a User of other services Multiple web services can be chained into an interactive workflow system The result is an agile application that can be created ‘just in time’ by the user for a specific need Service Broker Service Provider/User Publish Find Bind Service User Chain Service Provider Bind Chain

14. Features of the DVOY XML Web Service Architecture Interoperability: Platform and language independence; based on Web Services (XML,SVG) Legacy Support: Encapsulating existing data and exposing them as Web Services. (Access to standard HTTP/FTP servers) Just-in-time integration: Discovery, access to and ad-hoc chaining of services. (Future: Agile app building)

15. DVOY Web Services DVOY draws on three basic web services: DataCatalogWrapping service is for data registration, finding and wrapping information DataAccess service provides uniform access to heterogeneous, distributed, multidim. data DataPortrayal service prepares input data in form suitable for rendering

16. Applications…. Voyager Data Browser

17. Data Catalog All the data in the system are to be distributed on the Web and maintained by their custodians The purpose of the catalog is to help finding and and accessing the data Catalog would be limited to data that can be accessed/merged in DVOY

18. Catalog of Multidimensional Datasets Designed to Publish, Find(Select), Bind(Access) and Render distributed datasets Publishing is though an open web interface for user/broker registration of datasets Finding a dataset is aided by a metadata on Provider and Dataset Binding (data access) information is contained in the Dimensional Tables Rendering parameters are also contained in the Dimensional Tables

19. Geo-referenced Map Image: Image Indexed Variable Time, Var GeoRect, Var Sensor SatGeoImg CapImgIDUniversal ID ProviderURLURL SourceURLURL ImageURLURL ProviderAbbrevNASA GSFC ImageDescriptionText SensorEnumerated DateTime LatMin-65 LatMax 65 LonMin- 180 LonMax180 ImageWidthFor size verification ImageHeight MarginLeftFor trimming MarginTop MarginRight MarginBottom SQL Table … see table left NO DIMENSION TABLES XML Example: SeaWiFS

20. Geo-referenced Map Image: Time Indexed Regular Time Series; Fixed Georect; Image Trim < time_dimension handler_assembly="CAPITA.data.index.impl" handler_class="CAPITA.data.index.impl.datetime_interval" start_time="1996-07-25" end_time="yesterday"> sample_periodicity _unit = "day" sample_periodicity_mult = “1" TOMSFact TimeIndex Int, Calc from date DailyImage Img, Parsed from date LatMin Fixed, -65 LatMax Fixed, 65 LonMin Fixed, - 180 LonMax Fixed, 180 TOMSTime TimeIndex DateTime Virtual Tables (No physical StarTables)XML Example: TOMS ftp://jwocky.gsfc.nasa.gov/pub/eptoms/images/aerosol/y2002/ea020510.gif

21. Astronaut Photos: Image Indexed Variable Time, Image Center, Platform SatImg NASAImageIDUniversal ID ProviderURL http://eol.jsc.nasa.gov/ SourceURLhttp://eol.jsc.nasa.gov/script s/sseop/photo.pl?mission=IS S004&roll=E&frame=11079 ImageURLhttp://eol.jsc.nasa.gov/sseop/ images/ESC/small/ISS004/I SS004-E-11079.JPG ProviderAbbrevNASA JSC ImageDescriptionText PlatformISS004 GeoRegionGUATEMALA FeaturesSMOKE, GULF OF HONDURAS DateTime20020501 212700 Lat17.6 Lat-90.2 SQL Table <data_source service_url="http://capita.wustl.edu/dataservice/data.asmx" /> … see table left XML Example: ISS004

22. Atmospheric Photos: Image Indexed Variable Time, Image Center, Platform AtmoImg ImageIDUniversal ID ProviderURLhttp://capita.wustl.edu/capita /people/RHusar/rhusar.html SourceURLhttp://capita.wustl.edu/capita /people/RHusar/Pic/9812San FranciscoAGU/9812SanFra nciscoAGU.htm ImageURLhttp://capita.wustl.edu/capita /people/RHusar/Pic/9812San FranciscoAGU/Dcp01000E nh.jpg Provider AbbrevR Husar ImageDescriptionText PlatformAirplane GeoRegionNEVADA FeaturesDUST PLUME DateTime19981208 1800 Lat36.75 Lat-115.15 SQL Table <data_source service_url="http://capita.wustl.edu/dataservice/data.asmx" /> … see table left XML Example: Dust Plume Photo

23. DVOY State Maintenance This DVoyState data structure contains the state variables, which drive DVoy during browsing It is created by the DataCatalogWarpping service using the Catalog and Wrapping information User actions on Controllers modify the DVoyState structure directly but do not execute changes Actions are executed by managers (e.g. ConMan for connections) based on changes in DVoyState DVoyState resides on the client web page, hence it is unique for each Dvoy session The session state can be saved as a web page and can be recreated at load time Workbooks can be created from multiple pages of DVoyState (not yet implemented)

24. Dvoy Catalog Interaction: Find -> Bind WSGetDatasetList DatasetList.xmlDVoyState.xml WSGetDatasetInfo DatasetInfo.xml DatasetSelector DVoy Catalog Database On LoadCatalogPage On AddLayer Init DVoyStateAdd layer/dataset_abbr to DVoyStateAdd Layer Description to DVoyState DVoyState.xml

25. Dvoy Catalog Interaction: Find -> Bind DataSelectService – performs the FIND web service operation Queries the catalog for the list of datasets through GetDatasetList service Passes the returned DatasetList to a controller (listbox) where the user selects a dataset DataSelectService places selected DataCode into DVoyState.xml DataInfoService – performs the BIND web service operation Accepts data generic data request: GetAccessInfo (DatasetCode, Dimension, Param, Location, Time, ViewConstr) retrieves or generates access instructions for the query, places them into DVoyState.xml DataInfoService exposes the DataSelectService exposes the selected DataCode DatasetList DataInfoService AccessInfo DataSelectService DVoy Catalog Database On AddLayer DVoyState.xml GetDatasetList Browsing Environment CursorCodes DatasetCode Find Bind GetAccessInfo User User Selects New Data layer

26. DVOY Interfaces Data Input Data input Data Output - Browser The DVOY interface is composed of data viewers and controllers, all displayed on a webpage The web services and the preparation of the webpage interface is through.NET(Microsoft) The graphic data display on the webpage uses an SVG plugin (Adobe) The DVOY controls are linked to the SVG plugin and the.NET through client-side JavaScript Data Output – Web Service –The DVOY outputs are XML formatted datasets suitable for chaining with processing or rendering services

27. Event Handling in DVoy

28. Applications - Examples

30. Quebec Fires, July 6, 2002 SeaWiFS, METAR and TOMS Index superimposed SeaWiFS satellite and METAR surface haze shown in the Voyager distributed data browser Satellite data are fetched from NASA GSFC; surface data from NWS/CAPITA servers

31. Trans-Atlantic Transport of Quebec Smoke July 11: Smoke approaching Europe July 10: Quebec smoke over Mid- Atlantic SeaWiFS Reflectance TOMS Absorbing Aerosol SeaWiFS Reflectance TOMS Absorbing Aerosol Spain E. US

32. Web Service Wrapper for TOMS Aerosol Index Fixed Geo-rectangle; Image Trim, daily TOMS Image Metadata Geo-rectangle (65, -180; -65, 180) Image Size (640, 480) Image Margins (40, 40, 30,30) Transparent Colors (0,0,255 ) Image Access Metadata URL template: :ftp://jwocky.gsfc.nasa.gov/pub/eptoms/images/aerosol/YYYYY/eaYYMMDD.gif StartDate, EndDate

33. Browsing of Distributed Data from HTTP/FTP Servers Land Reflectance from SeaWiFS Project, NASA GSFC AVHRR Oceanic Aerosol CAPITA, WashU, STL Fire Pixels, Jan 1997, ESA Ionia Project

34. Voyager Spatio-Temporal Data Browser

35. NRL Forecast Model for Dust, Smoke and Sulfate METAR Surface Haze Real-time model and surface observations are compared spatially and temporally Dust Sulfate Smoke METAR Haze Time Selector

36. Current DVoy User Interface Components Catalog: User selects among distributed ‘wrapped’ data Map View: Displays Selected Data Control Panel: Provides user with facilities to customize display NOTE: The data sources are physically located on multiple databases across the country (NASA SQL, CIRA SQL, NOAA SQL …) Terry, the powerplant data could be handled in a similar fashion from the EPA Oracle database

37. Idaho Fires Example (August 2000) ASTR Fire Locations SeaWiFS Image showing smoke dispersion Satellite imagery and satellite fire observations are spatially integrated to show fire sources and smoke transport

38. Idaho Fires Example (August 2000) ASTR Fire Locations Processed SeaWiFS image showing aerosol optical depth Derived aerosol optical depth provides a focused look at smoke dispersion The SeaWiFS reflectance image is hidden to expose the aerosol optical depth image ‘below’ it

40. Overlaying PM2.5 Concentration Data http://capita.wustl.edu/dvoy_services/dvoy.aspx?layxml_file=IdahoFiresWithIMPROVEAug192000.xml PM2.5 data from the IMPROVE and AIRS database at Colorado State provide supporting evidence for smoke impacts Moving the cursor over the PM2.5 locations shows the lat, lon, and concentration for that monitor Live version at:

41. State-Aware, Bi-directional Client-Server Interaction With Perl and Client-Side JavaScript.State-Aware, Bi-directional Client-Server Interaction With Perl and Client-Side JavaScript. Ronan Oger ronan@roasp.com RO IT Systems GmbH Marktgasse 21 8001 Zurich Switzerland Keywords: Serverside SVG Generation; Thin Client; Perl; Client-Server Developing a scaleable, multi-browser, real-time arcade-style video game as an example of a thin-client state-aware application using Perl,SVG, and JavaScript The Internet provides near-instantaneous, content-rich communications between geographically distant agents. A key limiting factor on the usefulness of the Internet has until recently been the technological limitations on the type and direction of this data exchange. All transfers involving the HTTP protocol have been limited to half-duplex, unidirectional, stateless serial communications, or TCP-based communications between bandwidth-consuming plug-ins such as Applets, or security-prone, vendor-specific technologies such as ActiveX components. By combining server-side SVG generation using object-oriented languages such as Perl with client-side XML manipulation with JavaScript, content generators are now able to implement lightweight, bilateral, secure, low-bandwidth communication between the browser and the server. This enables the generation of state-aware, refresh-free web content, and the simultaneous manipulation of web content on multiple browsers or servers from a single browser or server. The integration of SVG into the Internet browser, along with the browser's support of the interaction between the SVG Document Object Model (DOM) and JavaScript, now provides the necessary foundation for rich, real-time, bi-directional client-server communication. The coming together of SVG, JavaScript, and Perl enable the browser to take its place on the computer desktop as a host for meaningful, graphically intense programs. By keeping the complex components of these programs on the server and limiting the processing complexity at the client to rendering, the SVG-Perl-JavaScript triangle has the potential for throwing into question the dominance of the Operating-System-resident application on the desktop, finally making it possible to build complete thin-client applications with a browser for the presentation layer and Perl on a server to for processing the business logic. The construction of a thin-client application will be explored through the example of an N-browser, interactive game using the powerful Perl SVG module family and JavaScript client-side event handlers. The exceptional qualities of Perl for SVG and XML Webservice applications will be demonstrated with a scaleable, self-contained, dedicated, platform-independent SVG Webservice application.

42. Dvoy Federated Information System Dvoy offers a homogeneous, read-only access mechanism to a dynamically changing collection of heterogeneous, autonomous and distributed information sources. Data access uses a global multidimensional schema consisting of spatial, temporal and parameter dimensions The uniform global schema is suitable for data browsing and online analytical processing, OLAP The limited global query capabilities yield slices along the spatial, temporal and parameter dimensions of the multidimensional data cubes.

43. Architecture of Dvoy Federated Information System After Busse et. al., 1999 The main software components of Dvoy are wrappers, which encapsulate sources and remove technical heterogeneity, and mediators, which resolve the logical heterogeneity. Wrapper classes are available for geo-spatial (incl. satellite) images, SQL servers, text files,etc. The mediator classes are implemented as web services for uniform data access to n-dimensional data.