Dvoy Related Ideas

Data Acquisition and Usage Value Chain

Data Processing Value Chain Monitor Store Data 1 Monitor Store Data 2 Monitor Store Data n Monitor Store Data m IntData1 IntDatan IntData2 Virtual Int. Data

Information Processing Value Chain (Taylor, 1975) Informing Knowledge ActionProductive Knowledge InformationData Organizing Grouping Classifying Formatting Displaying Analyzing Separating Evaluating Interpreting Synthesizing Judging Options Quality Advantages Disadvantages Deciding Matching goals, Compromising Bargaining Deciding Forces to Move Data one-shot to reusable form External force – contracts Internal – humanitarian, benefits Resistances to Move Data Mechanical Personal Institutional

DVOY (A Federated System for Finding, Exploring and Analyzing Environmental Data) (Unified Access to 4-Dimensional Geo-Environmental Data through Web Services) Outline Prepared by Special Interest Group on Environmental Data Integration March 2002 Coordinated by CAPITA Supported by NSF, EPA and NOAA

The Researcher’s Challenge “The researcher cannot get access to the data; if he can, he cannot read them; if he can read them, he does not know how good they are; and if he finds them good he cannot merge them with other data.” Information Technology and the Conduct of Research: The Users View National Academy Press, 1989 These resistances can be overcome through A catalog of distributed data resources for easy data ‘discovery’ Uniform data coding and formatting for easy access, transfer and merging Rich and flexible metadata structure to encode the knowledge about data Powerful shared tools to access, merge and analyze the data

Data Catalog All the data in the system are to be distributed on the Web and maintained by their custodians The purpose of the catalog is to help finding and and accessing the data Catalog would be limited to data that can be accessed/merged in DVOY

Uniform Coding and Formatting of Distributed Data Data are now easily accessible through standard Internet protocols, but the coding and formatting of the data is very heterogeneous On the other hand data sharing is most effective if the codes/formats/protocols are uniform (e.g. the Web formats and protocols ) Re-coding and reformatting all the heterogeneous data into universal form in their respective server is unrealistic An alternative is enrich the heterogeneous data with uniform coding along the way from the provider to the user. A third party ‘proxy’ server can perform the necessary homogenization with the following benefits: –The data user interfaces with a simple universal data query and delivery system (interface, formats..) –The data provider does not need to change the system; gets additional security protection since the data data accessed by the proxy –Reduced data flow resistances results in increased overall data flow and data usage.

OGC Web Service OGC Web Service Interoperability Program: GoalsGoals Promote interoperability. The interaction between services should be completely platform and language independent, based on XML Enable just-in-time integration. The discovery, access to and ad-hoc chaining of services should be possible dynamically at runtime. Reduce complexity. All components are services with published capabilities (incl. ConMan?); implementation is opaque. Support legacy systems. Enable interoperability by encapsulating existing components and exposing them as services. (Same as DVoy, isn’t it? We could put it better ourselves! RBH)

Outline of an Open, Distributed Air Quality Data Integration and Analysis System Notes prepared for a discussion with EPA NERL and OAQPS December 1, 1998

The Problem: The researcher is not aware of the relevant data; if he is aware, he can not access to them; if he can access them, he can not read them; if he can, he does not know how good they are; if they are good he cannot merge them with other data and by the time he merges them, the data are outdated Based on Information Technology and the Conduct of Research: The Users view National Academy Press, 1989

AQ Data and Analysis: Challenges and Opportunities Shift from primary to secondary pollutants. Ozone and PM2,5 travel miles across state or international boundaries and their sources are not well established New Regulatory approach. Compliance evaluation based on ‘weight of evidence’ and tracking the effectiveness of controls Shift from command & control to participatory management. Inclusion of federal, state, local, industry, international stakeholders. Challenges Broader user community. The information systems need to be extended to reach all the stakeholders ( federal, state, local, industry, international) A richer set of data and analysis. Establishing causality, ‘weight of evidence’, emissions tracking requires more data and air quality analysis Opportunities Rich AQ data availability. Abundant high-grade routine and research monitoring data from EPA, NASA, NOAA and other agencies are now available. New information technologies. DBMS, data exploration tools and web-based communication now allows cooperation (sharing) and coordination among diverse groups.

Challenges Broader user community. The information systems need to be extended to reach all the stakeholders ( federal, state, local, industry, international) A richer set of data and analysis. Establishing causality, ‘weight of evidence’, emissions tracking requires the analysis of air quality, meteorology emissions and effects data. Rich AQ data availability. Abundant high-grade routine and research monitoring data from EPA and other agencies are now available. New information technologies. DBMS, data exploration tools and web-based communication now allows cooperation (sharing) and coordination among diverse groups. Opportunities

Recap: Harnessing the Winds Secondary pollutants along with more open environmental management style are placing increasing demand on data analysis. Meanwhile, rich AQ data sets and the computer and communications technologies offer unique opportunities. It appears timely to consider the development of a web-based, open, distributed air quality data integration, analysis and dissemination system. The challenge is learn how to harness the winds of change as sailors have learned to use the winds for going from A to B

Standard Data Support System Data management systems, DBMS Data processing end exploration tools Presentation tools

Data Flow and Processing

Data sharing standards. A set of open standards for the sharing of AQ data, tools and reports. Examples: TCP/IP, HTML, XML, FGDC Data catalog. A virtual centralized catalog with search and retrieval facilities. Examples: GCMD, web-indexes Web-based shared workspace. Place to share comments, feedback, plans,... Infrastructure support for a distributed system

Benefits of a Distributed and Shared System Access to data. Users can get data, tools, reports out of the system for specific projects. It can be a forum for the exchange of ideas, peer- feedback etc. Saving time and money. The data, tools and other resources in the system could be leveraging the dollars and time available for specific projects. Recycling Data. Data are costly resource. The system can help managing, accessing and documenting one's own data, and share it with others for re-use.

The Dvoy Project DVOY is Federated Information System for heterogeneous, multidimensional datasets Voyager is a generic graphic browser for the federated DVOY data. The initial Dvoy infrastructure is being developed at CAPITA, with NSF supportDvoy Further services for data access, processing and viewing are expected from the community The project evolution is to ride 'web services wave‘ of the Internet CAPITA Support: –NSF ITRWorkgroup Collaboration Tool:Aug Aug 2004 –EPA Web-based Visibility:Aug Apr 2003 –NOAAASOS Visibility:Sep Sep 2002 –MARAMAChemical Trajectory Tool:Aug July 2003 –EPA OAQPS Global Transport Analysis:Nov 2002 – Oct 2003 –NSF DigiGov Fire and Smoke Network :May 2003 – Apr 2006 Pending –NASA ESE Satellite Appl. to PM Managment :June 2003 – May 2008 Pending In-kind support by organizations participating in DVOY-based federated data sharing Collaborators: CIRA (Schichtel), NRL(Westphal), NASA (Goddard)…many data sources.

DATAFED Catalog Maintenance System PUBLISHFINDVIEW