A Services Oriented Architecture for Water Resources Data David R. Maidment Center for Research in Water Resources University of Texas at Austin EPA Storet Conference Austin Tx, Nov 28, 2007

Collaborators University of Texas at Austin –Tim Whiteaker, Stephanie Johnson San Diego Supercomputer Center –Ilya Zaslavsky, David Valentine, Tom Whitenack Utah State University –David Tarboton, Jeff Horsburgh, Kim Schreuders Drexel University –Michael Piasecki, Bora Beran, Yoori Choi University of South Carolina –Jon Goodall

A Services Oriented Architecture for Water Resources Data CUAHSI and WATERS CUAHSI water web services WaterML and WQX Web Services for TMDL

A Services Oriented Architecture for Water Resources Data CUAHSI and WATERS CUAHSI water web services WaterML and WQX Web Services for TMDL

Ocean Sciences What is CUAHSI? CUAHSI – Consortium of Universities for the Advancement of Hydrologic Science, Inc Formed in 2001 as a legal entity Program office in Washington (5 staff) NSF supports CUAHSI to develop infrastructure and services to advance hydrologic science in US universities Earth Sciences Atmospheric Sciences UCAR CUAHSI Unidata HIS National Science Foundation Geosciences Directorate

CUAHSI Member Institutions 115 US Universities as of November 2007

Super Computer Centers: NCSA, TACC Domain Sciences: Unidata, NCAR LTER, CZEN GEON Government: USGS, EPA, NCDC Industry: ESRI, Kisters, Microsoft CUAHSI HIS HIS Team and its Cyberinfrastructure Partners HIS Team: Texas, SDSC, Utah, Drexel, Duke

Waters Network Testbed Sites

HIS Team WATERS Testbed WATERS Network Information System NSF has funded work at 11 testbed sites, each with its own science agenda. HIS supplies the common information system

Hydrologic Information Server Deployment National Hydrologic Information Server San Diego Supercomputer Center metadata for national datasets: NWIS, Storet, Snotel WATERS testbed server Provides access to 1246 sites in 16 observation networks

Hydrologic Information Server Microsoft SQLServer Relational Database Observations Data Geospatial Data GetSites GetSiteInfo GetVariables GetVariableInfo GetValues DASH – data access system for hydrologyWaterOneFlow services ArcGIS Server

Definition The CUAHSI Hydrologic Information System (HIS) is a geographically distributed network of data sources and functions that are integrated using a web services architecture so that they operate as a connected whole.

A Services Oriented Architecture for Water Resources Data CUAHSI and WATERS CUAHSI water web services WaterML and WQX Web Services for TMDL

Point Observations Information Model Data Source Network Sites Variables Values {Value, Time, Qualifier, Offset} Utah State Univ Little Bear River Little Bear River at Mendon Rd Dissolved Oxygen 9.78 mg/L, 1 October 2007, 6PM A data source operates an observation network A network is a set of observation sites A site is a point location where one or more variables are measured A variable is a property describing the flow or quality of water A value is an observation of a variable at a particular time A qualifier is a symbol that provides additional information about the value An offset allows specification of measurements at various depths in water GetSites GetSiteInfo GetVariables GetVariableInfo GetValues

CUAHSI Observations Data Model

Locations Variable Codes Date Ranges WaterML and WaterOneFlow GetSiteInfo GetVariableInfo GetValues WaterOneFlow Web Service Client STORET NAM NWIS Data Repositories Data EXTRACT TRANSFORM LOAD WaterML WaterML is an XML language for communicating water data WaterOneFlow is a set of web services based on WaterML

WaterOneFlow Set of query functions Returns data in WaterML Ilya Zaslavsky and David Valentine, SDSC

Data Heterogeneity Syntactic mediation –Heterogeneity of format –Use WaterML to get data into the same format Semantic mediation –Heterogeneity of meaning –Each water data source uses its own vocabulary –Match these up with a common controlled vocabulary –Make standard scientific data queries and have these automatically translated into specific queries on each data source

Search multiple heterogeneous data sources simultaneously regardless of semantic or structural differences between them Objective NWIS NARR NAWQA NAM-12 request request return return What we used to do ….. Michael Piasecki Drexel University

Semantic Mediator What we are doing now ….. NWIS NAWQA NARR generic request GetValues GetValues HODM Michael Piasecki Drexel University

Hydroseek Supports search by location and type of data across multiple observation networks including NWIS and Storet Bora Beran, Drexel

HydroTagger Ontology: A hierarchy of concepts Each Variable in your data is connected to a corresponding Concept

HIS to Google Earth developed by Peter Fitch, CSIRO, Australia A web application housed in Canberra, Australia, that operates over the WATERS Network data services

A Services Oriented Architecture for Water Resources Data CUAHSI and WATERS CUAHSI water web services WaterML and WQX Web Services for TMDL

The biggest difference is that the ODM concept of a time series data value (orange line) is derived from two sources: Field activity, and Analysis. This design is to simplify data management

SiteInfo Series GetSites GetSiteInfo Site and Series are captured in the WQX model. An activity can produce multiple results. This differs from the ODM model, where a series contains the results.

GetValues DataValues Method LabMethod SiteInfo Series GetValues would access all information (by design). Results captures multiple ODM concepts: DataValues, Method, and LabMethod

WaterOneFlow WQX Comparison WaterMLWQX WebServiceSingle WebService with basic methods. Designed for single calls to return the needed information set. Multiple WebServices each containing multiple methods for querying information XMLObjects/ Components consistent across methods Consistent, but users may need to make additional calls to methods to make an information set Multiple ResultNo web service methodSupported. Time SeriesBasis of designNot supported. Multiple results are the time series. Spatial QueryNo spatial query.Spatial queries by bounding box, and HUC code.

A Services Oriented Architecture for Water Resources Data CUAHSI and WATERS CUAHSI water web services WaterML and WQX Web Services for TMDL

Data Access System for Hydrology (DASH) Mission River (Water Quality Management Segment 2004) USGS Flow Gaging Station at Refugio Texas Web service to TCEQ Web service to USGS DASH map interface to associate gaging station with river segment

Web Services Spreadsheet for Load Duration Curves Web service for bacterial data Web service for flow data

Flow Duration Curve cfs Mission River at Refugio, Tx derived from USGS web service

Load Duration Curve (Fecal Coliform) Coliform Units per day TCEQ staff comment: “This tool produces $20,000 worth of graphs in 5 minutes”

Conclusion: Web services work! The CUAHSI Hydrologic Information System (HIS) is a geographically distributed network of hydrologic data sources and functions that are integrated using a web services architecture so that they function as a connected whole. For more information: