Waters Observatory Network Observe water systems Observe in many different contexts and for different purposes Have a Waters Network Information System for sharing data –Common language for data –Geographic federation of dispersed data sources through web services

Project co-PI in Phase 2 Collaborator in Phase I CUAHSI HIS Partner Institutions

10 WATERS Testbed Sites

HIS Team WATERS Testbed WATERS Network Information System

Super Computer Centers: NCSA, TACC Domain Sciences: Unidata, NCAR LTER, GEON Government: USGS, EPA, NCDC, USDA Industry: ESRI, Kisters, Microsoft CUAHSI HIS HIS Team and its Cyberinfrastructure Partners HIS Team: Texas, SDSC, Utah, Drexel, Duke

Super Computer Centers: NCSA TACC Domain Sciences: Unidata, NCAR LTER, GEON Government: USGS, EPA, NCDC, USDA Industry: ESRI, Kisters, Microsoft HIS Team WATERS Testbed WATERS Network Information System CUAHSI HIS HIS and WATERS

CUAHSI Member Institutions 116 Universities as of November 2006

Super computer Centers: NCSA, TACC Domain Sciences: Unidata, NCAR LTER, GEON Government: USGS, EPA, NCDC, USDA Industry: ESRI, Kisters, OpenMI HIS Team WATERS Testbed WATERS Network Information System CUAHSI HIS HIS, WATERS and the CUAHSI Community

KISTERS Water Resources Management Data System KISTERS is building CUAHSI web services onto their database Stefan Fuest

Super computer Centers: NCSA, TACC Domain Sciences: Unidata, NCAR LTER, GEON Government: USGS, EPA, NCDC, USDA Industry: ESRI, Kisters, OpenMI HIS Team WATERS Testbed WATERS Network Information System CUAHSI HIS International Partners CSIRO Land and Water Resources Water Resources Observations Network (WRON) European Commission Water database design and model integration (HarmonIT and OpenMI)

HIS Goals Hydrologic Data Access System – better access to a large volume of high quality hydrologic data Support for Observatories – synthesizing hydrologic data for a region Advancement of Hydrologic Science – data modeling and advanced analysis Hydrologic Education – better data in the classroom, basin-focused teaching

Rainfall & Snow Water quantity and quality Remote sensing Water Data Modeling Meteorology Soil water

Water Data Web Sites

NWISWeb site output # agency_cd Agency Code # site_no USGS station number # dv_dt date of daily mean streamflow # dv_va daily mean streamflow value, in cubic-feet per-second # dv_cd daily mean streamflow value qualification code # # Sites in this file include: # USGS NEUSE RIVER NEAR CLAYTON, NC # agency_cdsite_nodv_dtdv_vadv_cd USGS USGS USGS USGS USGS USGS USGS USGS USGS USGS USGS Time series of streamflow at a gaging station USGS has committed to supporting CUAHSI’s GetValues function

Observation Stations Ameriflux Towers (NASA & DOE)NOAA Automated Surface Observing System USGS National Water Information SystemNOAA Climate Reference Network Map for the US

Water Quality Measurement Sites in EPA Storet Substantial variation in data availability from states Data from Bora Beran, Drexel University

Water Quality Measurement Sites from Texas Commission for Environmental Quality (TCEQ)

Geographic Integration of Storet and TCEQ Data in HIS

Observations Catalog Specifies what variables are measured at each site, over what time interval, and how many observations of each variable are available

Point Observations Information Model Data Source Network Sites Variables Values {Value, Time, Qualifier} USGS Streamflow gages Neuse River near Clayton, NC Discharge, stage (Daily or instantaneous) 206 cfs, 13 August 2006 A data source operates an observation network A network is a set of observation sites A site is a point location where one or more variables are measured A variable is a property describing the flow or quality of water A value is an observation of a variable at a particular time A qualifier is a symbol that provides additional information about the value

Data Discovery and Delivery Data Source Network Sites Variables Values Observations metadata Observations data HIS Server Observations Catalog Web services Data Discovery Data Delivery HIS facilitates data discovery by building and maintaining observations catalogs Data delivery occurs through web services from remote data archives or local observations databases. Water resource agencies support data delivery services.

NWIS ArcGIS Excel NCAR Unidata NASA Storet NCDC Ameriflux Matlab AccessJava Fortran Visual Basic C/C++ Some operational services CUAHSI Web Services Data Sources Applications Extract Transform Load

WaterOneFlow Web Services Data accessthrough web services Data storage through web services Downloads Uploads Observatory servers Workgroup HIS SDSC HIS servers 3 rd party servers e.g. USGS, NCDC GIS Matlab IDL Splus, R D2K, I2K Programming (Fortran, C, VB) Web services interface Web portal Interface (HDAS) Information input, display, query and output services Preliminary data exploration and discovery. See what is available and perform exploratory analyses HTML -XML WSDL - SOAP Hydrologic Information System Service Oriented ArchitectureNWIS ArcGIS Excel NCAR Unidata NASA Storet NCDC Ameriflux Matlab AccessSAS Fortran Visual Basic C/C++ CUAHSI Web Services Ilya Zaslavsky

My server Web services: key terms Web Service Code WSDL Web service description language Your Web application Reference a web service, and call a WS method Your desktop application Reference a web service, and call a WS method SOAP Messages SOAP: Simple Object Access Protocol

- From different database structures, data collection procedures, quality control, access mechanisms  to uniform signatures … Water Markup Language - Tested in different environments - Standards-based - Can support advanced interfaces via harvested catalogs - Accessible to community - Templates for development of new services - Optimized, error handling, memory management, versioning, run from fast servers And: working with agencies on setting up services!

Web service testing pages

For Developers and Information Managers Code developers –HIS Team System: code repository, web services –Design Documents –Workbook for using web services in different environments –Web service testing and versioning guidelines and modules –Support Site managers –Step-by-step instructions for setting up the server (eventually, will be an “HIS appliance”) –Configuration and data management tools What it takes to setup and maintain HIS server, and how to get there from your current setup – THIS AFTERNOON

CUAHSI Hydrologic Information System Levels National HIS – San Diego Supercomputer Center Workgroup HIS – research center or academic department Personal HIS – an individual hydrologic scientist HIS Server HIS Analyst Map interface, observations catalogs and web services for national data sources Map interface, observations catalogs and web services for regional data sources; observations databases and web services for individual investigator data Application templates and HydroObjects for direct ingestion of data into analysis environments: Excel, ArcGIS, Matlab, programming languages; MyDB for storage of analysis data

Data Ingestion MatlabExcelArcGISJava.Net HydroObjects HIS Analyst HIS Server WaterOneFlow Services Observations Data Tim Whiteaker

Tutorials and Templates that show how to use Web services in various applications ftp://ftp.crwr.utexas.edu/pub/outgoing/CUAHSI/HIS_workbook/ / Includes HydroObjects Library

HIS Analyst Demo (Tim Whiteaker) Ingestion of data using web services into Excel and VB.Net

CUAHSI Hydrologic Data Access System A common data window for accessing, viewing and downloading hydrologic information USGS NASANCDC EPANWS Observatory Data

HIS Server Supports data discovery, delivery and publication –Data discovery – how do I find the data I want? Map interface and observations catalogs –Data delivery – how do I acquire the data I want? Use web services or retrieve from local database –Data Publication – how do I publish my observation data? Use Observations Data Model

HIS Server Architecture Map front end – ArcGIS Server 9.2 (being programmed by ESRI Water Resources for CUAHSI – led by Dean Djokic, an IHE graduate) Relational database – SQL/Server 2005 or Express Web services library – VB.Net programs accessed as a Web Service Description Language (WSDL)

HIS Server Demo (Dean Djokic) ESRI Water Resources Applications Group, Redlands CA (a voluntary contribution to CUAHSI)

Digital Watershed How can hydrologists integrate observed and modeled data from various sources into a single description of the environment?

Digital Watershed Hydrologic Observation Data Geospatial Data Weather and Climate Data Remote Sensing Data (NetCDF) (GIS) (Relational database) (EOS-HDF) Digital Watershed A digital watershed is a synthesis of hydrologic observation data, geospatial data, remote sensing data and weather and climate data into a connected database for a hydrologic region

HIS Servers Hydrologic Observations Server GIS Data Server Weather and Climate Server Remote Sensing Server Digital Watershed HIS Servers provide hydrologic observations, weather and climate, GIS and remote sensing data. For HIS version 1.0, the focus is a hydrologic observations server for data from gages and monitoring sites at point locations.

Water Resource Regions and HUC’s

NHDPlus for Region 17E

NHDPlus Reach Catchments ~ 3km 2 About 1000 reach catchments in each 8-digit HUC Average reach length = 2km 2.3 million reaches for continental US

National Land Cover Dataset the data:

Reach Attributes Slope Elevation Mean annual flow –Corresponding velocity Drainage area % of upstream drainage area in different land uses Stream order

1:250,000 Scale Soil Information

Hydrologic Landscape Regions USGS Characterization of nature of soil and groundwater systems

Arc Hydro: GIS for Water Resources Arc Hydro –An ArcGIS data model for water resources –Arc Hydro toolset for implementation –Framework for linking hydrologic simulation models The Arc Hydro data model and application tools are in the public domain Published in 2002

Arc Hydro Links Space and Time Integrate geospatial data for land and water systems You store this information in a relational database or get it from web services Link space and time with HydroID All Hydrofeatures of a testbed area will have a unique HydroID Land base from NHDPlus + added data layers

Open Hydro adopts the Arc Hydro data model and exposes it using open source GIS and database tools. Open Hydro also includes a suite of tools for working with hydrologic data and managing the associated hydro geodatabase. Dan Ames, Idaho State University

Based on MapWindow GIS: Free open source GIS alternative for data visualization and analysis. MapWindow GIS follows OGC standards and works with ESRI data formats. MapWindow GIS is extensible using VB.NET, VB6, C#, and ActiveX. Sponsored by NASA, NOAA, DOE, and EPA (new platform for BASINS). Nearly 30,000 downloaders so far at:

Open Hydro is still under development and ISU is seeking partners to work together on various functions. Current completed functions include geodatabase import/export, and automatic watershed delineation and support for ESRI’s geography network and online data sources.

Demo of NHDPlus and Weather Downloader for the San Marcos Basin By Ernest To Center for Research in Water Resources

Search multiple heterogeneous data sources simultaneously regardless of semantic or structural differences between them Objective NWIS NARR NAWQA NAM-12 request request return return What we are doing now ….. Michael Piasecki Drexel University

Semantic Mediator What we would like to do ….. NWIS NAWQA NARR generic request GetValues GetValues HODM Michael Piasecki Drexel University

Demo on Semantic Mediator By Bora Beran and Michael Piasecki, Duke University

Project sponsored by the European Commission to promote integration of water models within the Water Framework Directive Software standards for model linking Uses model core as an “engine”

OpenMI – Links Data and Simulation Models CUAHSI Observations Data Model as an OpenMI component Simple River Model Trigger (identifies what value should be calculated)

Typical model architecture Application User interface + engine Engine Simulates a process – flow in a channel Accepts input Provides output Model An engine set up to represent a particular location e.g. a reach of the Thames Engine Output data Input data Model application Run Write Read User interface

AcceptsProvides Rainfall (mm) Runoff (m 3 /s) Temperature (Deg C) Evaporation (mm) AcceptsProvides Upstream Inflow (m 3 /s) Outflow (m 3 /s) Lateral inflow (m 3 /s) Abstractions (m 3 /s) Discharges (m 3 /s) River Model Linking modelled quantities

Data transfer at run time Rainfall runoff Output data Input data User interface River Output data Input data User interface GetValues(..)

Models for the processes River (InfoWorks RS) Rainfall (database) Sewer (Mouse) RR (Sobek-Rainfall -Runoff)

Data exchange 3 Rainfall.GetValues River (InfoWorks-RS) Rainfall (database) Sewer (Mouse) 2 RR.GetValues 7 RR.GetValues RR (Sobek-Rainfall -Runoff) 1 Trigger.GetValues 6 Sewer.GetValues call data

OpenMI OpenMI = Open Modeling Interface It is a Standard for Model Linking It is part of HarmonIT, a European Commission funded research project with 14 project partners (universities & software companies)

OpenMI Conceptual Framework VALUES All values are referenced in a what-where-when framework, allowing different data resources or models to communicate data

HIS as OpenMI Components water balance model NWIS Streamflow Daymet Precipitation To calculate storage, the model needs inflow and outflow To calculate storage, the model needs precipitation Goal: Link the National HIS web services with a simple water balance model using OpenMI as the mediator Trigger: Calculate storage

Demo on Intregating Web Services and OpenMI to make a water balance model By Jon Goodall, Duke University

Watershed Hydrovolumes Geovolume is the portion of a hydrovolume that contains solid earth materials USGS Gaging stations Hydrovolume

Stream channel Hydrovolumes Need the capacity to represent Acoustic Doppler Current Profiler (ADCP) data (Iowa) Residence time distributions (Minnesota)

Integration of surface water and groundwater data Describe the relationship between surface water features ( e.g. streams and waterbodies) with groundwater features (aquifers, wells). Enable the connection with the surface water data model Hydro networkAquifers In the future go to 3D...

Water OneFlow Like Geospatial OneStop, we need a “Water OneFlow” – a common window for water data and models Advancement of water science is critically dependent on integration of water information Federal Academic Local State