Web Services Implementation Case Study: DataFed Air Quality Data & Services Project Coordinators: Software Architecture: R. Husar Software Implementation:

Slides:

Advertisements

Similar presentations

REASoN REASoN Project to link NASA's data, modeling and systems to users in research, education and applications Application of NASA ESE Data and Tools.

Advertisements

OGC Demo at IGARSS06 July 30 - Denver, CO Telecon 11 July 2006 Liping Di, George Mason University Rudolf Husar, Washington University.

Web 2.0 in Air Quality Analysis and Management Rudolf B. Husar and Erin M. Robinson Washington University, St. Louis, MO 2007 National Environmental Information.

DataFed and FASTNET Tools for Agile Air Quality Analysis Husar & Poirot.

The Frontier of GIS GIS Web Services Nadine Alameh, Global Science & Technology Next Generation of Community Statistical Systems Tampa, Florida March 14,

ESIP Air Quality Workgroup and the GEO Air Quality Community of Practice collaboratively building an air quality community network for finding, accessing,

Stefan Falke Center for Air Pollution Impact and Trend Analysis Washington University in St. Louis Networked Data and Tools for Environmental Management.

Distributed Voyager (DVoy) Web Services

DRAFT June 6, 2005 ESIP AQ Cluster, Air Quality Cluster Air Quality Cluster TechTrack Earth Science Information Partners Partners NASA.

2009 Progress Report to UIC: GEOSS Air Quality Community of Practice (AQ CoP link to cop site ) Rudolf Husar (Point of Contact, Washington.

Instrument Builders Information Specialists (ESIP) Scientists Curriculum Developers Teachers Decision Analysts Decision Makers Reports From Kim Kastens.

REASoN REASoN Project to link NASA's data, modeling and systems to users in research, applications and education Application of NASA ESE Data and Tools.

Architecture and Technologies for an Agile, User-Oriented Air Quality Data System Rudolf B. Husar Washington University, St. Louis Presented at the workshop.

Federated Network for Sharing Air Quality Data and Processing Services Center for Air Pollution Impact and Trend Analysis (CAPITA) Washington University,

Work Group Meeting on HTAP-Relevant IT Techniques, Tools and Philosophies: DataFed Experience and Perspectives Rudolf B. Husar CAPITA, Washington University,

1 The FASTNET Project Presented by: Sean Raffuse 1 Rudy Husar 2 Rich Poirot and Gary Kleiman 3 1 Sonoma Technology, Inc. 2 Center for Air Pollution Impact.

REASoN REASoN Project to link NASA's data, modeling and systems to users in research, education and applications Application of NASA ESE Data and Tools.

Spatio-Temporal Data Sharing using XML Web Services Presented at the Workgroup Meeting on Web-based Environmental Information System for Global Emission.

Current Air Quality Information ‘Ecosystem’ (Draft for Feedback) AQ information includes emissions, ambient & satellite data and model outputs The distributed.

Stefan Falke Center for Air Pollution Impact and Trend Analysis Washington University in St. Louis Brooke Hemming US EPA – Office of Research and Development.

Application of ESE Data and Tools to Particulate Air Quality Management The CAPITA REASoN Project August 15, 2003 Stefan Falke and Rudolf Husar Center.

Air Quality Cluster Air Quality Cluster TechTrack Earth Science Information Partners Partners(?) NASA NOAA EPA USGS DOE NSF Industry… Data Flow Technologies.

Accessing and Using Fire-Related Data with the CAPITA DataFed.net* Services Framework Stefan Falke Rudolf Husar Kari Hoijarvi Washington University in.

Air Quality Data Services: Application of OGC specifications Air Quality Data: Multi-dimensional, multi-source, multi-format Point observations are collected.

1 Application Scenario: Smoke Impact REASoN Project: Application of NASA ESE Data and Tools to Particulate Air Quality Management (PPT/PDF)Application.

Current Project Objectives The project’s focus is on criteria pollutants and toxics because of their availability and accessibility. Recommend and demonstrate.

OGC Demo at IGARSS06 July 30 - Denver, CO Telecon 11 July 2006 Liping Di, George Mason University Rudolf Husar, Washington University.

1 Using the GEOSS Common Infrastructure in the Air Quality & Health SBA: Wildfire & Smoke Assessment Prepared by the GEOSS AIP-2 Air Quality & Health Working.

Select, Overlay, Explore; Integration of diverse data Distributed Data Heterogeneous coding, access Connects providers to users; Homogenize data access.

Stefan Falke and Rudolf Husar Center for Air Pollution Impact and Trend Analysis Washington University in St. Louis A NSF Digital Government Pilot Project.

Enabling Tools and Methods for International, Inter- disciplinary and Educational Collaboration E. M. Robinson, K. Hoijarvi, S. Falke, E. Fialowski, M.

Federated Network for Sharing Air Quality Data and Processing Services Center for Air Pollution Impact and Trend Analysis (CAPITA) Washington University,

Part I DataFed An Agile Distributed Air Quality Data System Rudolf B. Husar Washington University, St. Louis Seminar Presented at University of Alabama,

COMMUNITY. Data Acquisition and Usage Value Chain.

Dvoy Related Ideas. Data Acquisition and Usage Value Chain.

Fire Emissions Network Sept. 4, 2002 A white paper for the development of a NSF Digital Government Program proposal Stefan Falke Washington University.

NASA REASoN Project SHAirED: S ervices for H elping the Air -quality Community use E SE D ata Stefan Falke, Kari Höijärvi and Rudolf Husar, Washington.

NASA REASoN Project SHAirED: S ervices for H elping the Air -quality Community use E SE D ata Stefan Falke, Kari Höijärvi and Rudolf Husar, Washington.

2009 Progress Report to UIC: GEOSS Air Quality Community of Practice (AQ CoP link to cop site ) Rudolf Husar (Point of Contact, Washington.

Processes of the Information Value Chain Informing Knowledge ActionProductive Knowledge Information Organizing Grouping Classifying Formatting Geo-referencing.

Part I DataFed An Agile Distributed Air Quality Data System Rudolf B. Husar Washington University, St. Louis Seminar Presented at University of Alabama,

Architecture and Technologies for an Agile, User-Oriented Air Quality Data System Rudolf B. Husar Washington University, St. Louis Presented at the workshop.

Architecture and Technologies for an Agile, User-Oriented Air Quality Data System Rudolf B. Husar Washington University, St. Louis Presented at the workshop.

Web Services-Based Mediator of Distributed Data Flow and Processing Project Coordinators: Software Architecture: R. Husar Software Implementation: K. Höijärvi.

An Integrated Fire, Smoke and Air Quality Data & Tools Network Stefan Falke and Rudolf Husar Center for Air Pollution Impact and Trend Analysis Washington.

1 Using the GEOSS Common Infrastructure in the Air Quality & Health SBA: Wildfire & Smoke Assessment Prepared by the GEOSS AIP-2 Air Quality & Health Working.

ESIP Air Quality Jan Air Quality Cluster Air Quality Cluster Technology Track Earth Science Information Partners Partners NASA NOAA EPA (?) USGS.

: Data Sharing/Processing Infrastructure Data Catalog and Access Dozens of datasets on aerosols, emissions, fire, meteorology,

The Federated Data System DataFed: Experiences in Data Homogenization and Networking R.B. Husar, K. Hoijarvi, S. R. Falke, E. M. Robinson, Washington University,

Copyright 2007, Information Builders. Slide 1 iWay Web Services and WebFOCUS Consumption Michael Florkowski Information Builders.

1 SEEDS IT Vision Scenario: Smoke Impact REASoN Project: Application of NASA ESE Data and Tools to Particulate Air Quality Management (PPT/PDF)Application.

Concepts on Aerosol Characterization R.B. Husar Washington University in St. Louis Presented at EPA – OAQPS Seminar Research Triangle Park, NC, April 4,

DRAFT June 6, 2005 ESIP AQ Cluster, Contact R. Husar Air Quality Cluster Air Quality Cluster TechTrack Earth Science Information Partners.

Application of NASA ESE Data and Tools to Particulate Air Quality Management A proposal to NASA Earth Science REASoN Solicitation CAN-02-OES-01 REASoN:

The Federated Data System DataFed R. Husar, K. Hoijarvi, S. Falke, DaFed Community EPA Data Summit, Feb. 12, 2008, RTP Non-intrusive data integration infrastructure.

Design and Implementation of a Distributed Data Exploration and Processing System by Marin Bezic Presented as a partial fulfillment of the requirements.

Standards-based Access to Air Quality Data: Application of OGC WMS and WCS Protocols Client Server Std. Interface GetCapabilities GetData Capabilities,

Voyager Data Services Services for Finding, Exploring and Presenting Distributed Environmental Data Outline Prepared by Voyager Interest Group on Environmental.

Federated Network for Sharing Air Quality Data and Processing Services Center for Air Pollution Impact and Trend Analysis (CAPITA) Washington University,

HTAP Data Network GEO Task DA-09-02d: Atm. Model Evaluation Network Project Officer: Terry Keating, EPA ESIP Meeting, Air Quality Workgroup January 5,

Fire, Smoke & Air Quality: Tools for Data Exploration & Analysis : Data Sharing/Processing Infrastructure This project integrates.

Online Software November 10, 2009 Infrastructure Overview Luciano Orsini, Roland Moser Invited Talk at SuperB ETD-Online Status Review.

There is increasing evidence that intercontinental transport of air pollutants is substantial Currently, chemical transport models are the main tools for.

NATIONAL AERONAUTICS AND SPACE ADMINISTRATION ESDS Reuse Working Group Earth Science Data Systems Reuse Working Group Case Study: SHAirED Services for.

DATAFED Application Programs. Dvoy Data Flow and Processes DataView 1 View Data Abstract Portrayal Device Portrayal Render Device View Portrayal Device.

ESIP Air Quality Jan Air Quality Cluster Air Quality Cluster Technology Track Earth Science Information Partners Partners NASA NOAA EPA (?) USGS.

MetaOS Concept MetaOS developed by Ambient Computing to coordinate the function of smart, networked devices Smart networked devices include processing.

GEOSS Air Quality Community Infrastructure

The GEO Air Quality Community of Practice

4/5 May 2009 The Palazzo dei Congressi di Stresa Stresa, Italy

Presentation transcript:

Web Services Implementation Case Study: DataFed Air Quality Data & Services Project Coordinators: Software Architecture: R. Husar Software Implementation: K. Höijärvi Data and Applications: S. Falke, R. Husar Center for Air Pollution Impact and Trend Analysis (CAPITA) Washington University, St. Louis, MO April 2005,

Agenda Project Overview Motivation Approach Web Services Provided WS Implementation Challenges WS Benefits Conclusion

DataFed Overview Vision –Better air quality management and science through effective use of relevant data Goals –Facilitate flow of atmospheric data from provider to users –Support development of user-driven data processing value chains –Participate in specific application projects Approach –Building WS-based data gateway & source wrapper utilities –Building WS-based browsers and analysis tools Data Provided –Air quality Services Provided –Subsetting, fusion/layering URL –

Motivation: Air Quality Management Requires a More Agile Information System Challenges –Shift from primary to secondary pollutants –New regulatory approach based on weight of evidence –Shift from command & control to participatory management –Broader user community –A richer set of data and analysis Opportunities –Rich AQ data availability from EPA, NASA, NOAA available. –New information technologies Monitoring Controls Set Policy Set Goals Assess- ment Air Quality

Approach: Giving Users Control Data are supplied by the provider and exposed on the smorgasbord Data selection and tailoring choices are made by the user Provider Push User Pull Flow of Data Flow of Control AQ DATA METEOROLOGY EMISSIONS DATA Informing Public AQ Compliance Status and Trends Network Assess. Tracking Progress Data to Knowledge Transformation Data Refining & Mediation Services

DataFed Services Software for the User –Data catalog –Dataset Viewer/Editor –Web page collection of views (console) –Predefined web service chains (mini-apps) Software for the Developer –*Dataset registration –*Data access –*Data processing –*Data rendering tasks –Web service chaining facility –Dataset wrapper generator * Available as a web service

DataFed Services Example: Subsets/Views Views are slices through a cube of data organized by lat, lon, altitude, and time (X,Y,Z,T)

WS Implementation Challenges Providing access via web services to data on web pages –Useful data is often embedded in web pages –Service interface to data source would be helpful –Developed wrapper to clip data from web pages {List others here} Need client/GUI to demonstrate WS benefits? Reliability: Distributed computing issues: network reliability, bandwidth, etc Chaining: Orchestrating distributed web services to act as a single application Links: Linking users to providers and other federations (e.g. OGC, OPenDAP)

Implementation Challenge: Providing Access via Web Services to Data on Web Pages Short descriptor defines exact location of data on page Wrapper software uses descriptor to present data on web page as a web service src_img_width src_img_height src_margin_rightsrc_margin_left src_margin_top src_margin_bottom src_lon_min src_lat_max src_lat_min src_lon_max Image Description for Data Access: src_image_width=502 src_image_height=329 src_margin_bottom=105 src_margin_left=69 src_margin_right=69 src_margin_top=46 src_lat_min=-70 src_lat_max=70 src_lon_min=-180 src_lon_max=180 Transparent colors for overlays RGB(89,140,255) RGB(41,117,41) RGB(23,23,23) RGB(0,0,0)

Reflectance (SeaWiFS) Organics (IMPROVE) Air Trajectory (ATAD) Map Border (OGC) Web Service Benefits Data from web services can be easily tailored and combined with other data

Web Service Benefits: Creating a Custom View From Multiple Layers in a Page of XML Name & title the new view Identify each of 4 layers

Implementation Challenge: Providing a GUI for WS Information Core maintains the state and executes the data selection, access and render services Wrappers encapsulate the heterogeneous external data sources Device Drivers translate generic, abstract graphic objects to specific devices and formats Ports connect the internal parameters of the program to external controls WDSL web service description documents Data Sources Controls Displays I/O Layer Device Drivers Wrappers App State Data Flow Interpreter Core Web Services WSDL Ports

Conclusions DatFed is an infrastructure for non-intrusively federating autonomous, distributed, heterogeneous datasets Data are wrapped for access through SOAP web services Service chaining performs filtering, aggregation, fusion, overlay, rendering and other operations on data. Several successful web-applications were built from the federated data and service components. A key challenge is to link DataFed to other open data systems (e.g. OpenDAP, OGC) and to interoperate with other processing services.