David R. Maidment (PI) David G Tarboton Ilya Zaslavsky Michael Piasecki Jon Goodall With support from collaborators, postdocs and graduate students: Rick Hooper, Jon Duncan, David Valentine, Tom Whitenack, Jeff Horsburgh, Bora Beran, Tim Whiteaker, Ernest To, Cedric David CUAHSI Hydrologic Information System Update

Definition The CUAHSI Hydrologic Information System (HIS) is a geographically distributed network of hydrologic data sources and functions that are integrated using web services so that they function as a connected whole.

CUAHSI HIS Update Current Status of the Project HIS 1.0 Components –WaterOneFlow web services –Observations data model Digital Watershed

CUAHSI HIS Update Current Status of the Project HIS 1.0 Components –WaterOneFlow web services –Observations data model Digital Watershed

HIS Progress Phase 1 grant, 2.5 year grant completed Phase 2 grant, 5-yr, $4.5M to Maidment (Texas) Co-PI’s –Ilya Zaslavsky (San Diego Supercomputing Center) –David Tarboton (Utah State) –Michael Piasecki (Drexel) –Jon Goodall (Duke) WATERS Testbeds serve as beta-testers for software First community-wide distribution in 12 – 18 months (dependant on establishing support services)

Project co-PI in Phase 2 Collaborator in Phase I CUAHSI HIS Components (in development)

Super Computer Centers: NCSA, TACC Domain Sciences: Unidata, NCAR LTER, GEON Government: USGS, EPA, NCDC, USDA Industry: ESRI, Kisters, Microsoft CUAHSI HIS HIS Team and its Cyberinfrastructure Partners HIS Team: Texas, SDSC, Utah, Drexel, Duke

HIS Team WATERS Testbed WATERS Network Information System

Super computer Centers: NCSA, TACC Domain Sciences: Unidata, NCAR LTER, GEON Government: USGS, EPA, NCDC, USDA Industry: ESRI, Kisters, OpenMI HIS Team WATERS Testbed WATERS Network Information System CUAHSI HIS HIS, WATERS and the CUAHSI Community

Super computer Centers: NCSA, TACC Domain Sciences: Unidata, NCAR LTER, GEON Government: USGS, EPA, NCDC, USDA Industry: ESRI, Kisters, OpenMI HIS Team WATERS Testbed WATERS Network Information System CUAHSI HIS International Partners CSIRO Land and Water Resources Water Resources Observations Network (WRON) European Commission Water database design and model integration (HarmonIT and OpenMI)

CUAHSI HIS Update Current Status of the Project HIS 1.0 Components –WaterOneFlow web services –Observations data model Digital Watershed

Rainfall & Snow Water quantity and quality Remote sensing Water Data Modeling Meteorology Soil water

Water Data Web Sites

NWISWeb site output # agency_cd Agency Code # site_no USGS station number # dv_dt date of daily mean streamflow # dv_va daily mean streamflow value, in cubic-feet per-second # dv_cd daily mean streamflow value qualification code # # Sites in this file include: # USGS NEUSE RIVER NEAR CLAYTON, NC # agency_cdsite_nodv_dtdv_vadv_cd USGS USGS USGS USGS USGS USGS USGS USGS USGS USGS USGS Time series of streamflow at a gaging station USGS has committed to supporting CUAHSI’s GetValues function

Observation Stations Ameriflux Towers (NASA & DOE)NOAA Automated Surface Observing System USGS National Water Information SystemNOAA Climate Reference Network Map for the US

Water Quality Measurement Sites in EPA Storet Substantial variation in data availability from states Data from Bora Beran, Drexel University

Water Quality Measurement Sites from Texas Commission for Environmental Quality (TCEQ)

Geographic Integration of Storet and TCEQ Data in HIS

Observations Catalog Specifies what variables are measured at each site, over what time interval, and how many observations of each variable are available

CUAHSI Hydrologic Data Access System A common data window for accessing, viewing and downloading hydrologic information USGS NASANCDC EPANWS Observatory Data (being built using HIS Server in collaboration with ESRI)

HIS Server Supports data discovery, delivery and publication –Data discovery – how do I find the data I want? Map interface and observations catalogs Metadata based Search –Data delivery – how do I acquire the data I want? Use web services or retrieve from local database –Data Publication – how do I publish my observation data? Use Observations Data Model

HIS Server and Analyst HIS Server Implemented at San Diego Supercomputer Center and at academic departments and research centers Implemented by individual hydrologic scientists using their own analysis environments HIS Analyst Web Services Sustainable – industrial strength technology Flexible – any operating system, model, programming language or application Details of HIS Analyst are here

Point Observations Information Model Data Source Network Sites Variables Values {Value, Time, Qualifier} USGS Streamflow gages Neuse River near Clayton, NC Discharge, stage (Daily or instantaneous) 206 cfs, 13 August 2006 A data source operates an observation network A network is a set of observation sites A site is a point location where one or more variables are measured A variable is a property describing the flow or quality of water A value is an observation of a variable at a particular time A qualifier is a symbol that provides additional information about the value

Example: Matlab use of CUAHSI Web Services % create NWIS Class and an instance of the class createClassFromWsdl(' w/NWIS/DailyValues.asmx?WSDL'); WS = NWISDailyValues; % Site Info for Site of Interest siteid='NWIS: '; strSite=GetSiteInfoObject(WS,siteid,''); strSite.site.siteInfo.siteName ans = NEUSE RIVER NEAR CLAYTON, NC lat=strSite.site.siteInfo.geoLocation.geogLocation.lat itude long=strSite.site.siteInfo.geoLocation.geogLocation.lo ngitude lat = long =

Variable and variableTimeInterval strSite.site.seriesCatalog(1).series(:).variable ans = variableCode: '00065' variableName: 'Gage height, feet' units: 'international foot' ans = variableCode: '00060' variableName: 'Discharge, cubic feet per second' units: 'cubic feet per second' strSite.site.seriesCatalog(1).series(:).variableTimeInt erval ans = beginDateTime: ' T00:00:00' endDateTime: ' T00:00:00' ans = beginDateTime: ' T00:00:00' endDateTime: ' T00:00:00'

getVariableInfo varcode='NWIS:00060'; varInfo=GetVariableInfoObject(WS,varcode,'') varInfo = variables: [1x1 struct] varInfo.variables.variable ans = variableCode: '00060' variableName: 'Discharge, cubic feet per second' units: 'cubic feet per second'

GetValues % GetValues to get the data siteid='NWIS: '; bdate=' T00:00:00'; edate=' T00:00:00'; variable='NWIS:00060'; valuesxml=GetValues(WS,siteid,variable,bdate,edate,'');

Parse XML and Analyze % Parse the XML into a Matlab object to work with valuesobj=xml_parseany(valuesxml);... plot(date,flowval);datetick;

NWIS ArcGIS Excel NCAR Unidata NASA Storet NCDC Ameriflux Matlab AccessJava Fortran Visual Basic C/C++ Some operational services CUAHSI Web Services Data Sources Applications Extract Transform Load

Search multiple heterogeneous data sources simultaneously regardless of semantic or structural differences between them Objective NWIS NARR NAWQA NAM-12 request request return return What we are doing now ….. Michael Piasecki Drexel University

Semantic Mediator What we would like to do ….. NWIS NAWQA NARR generic request GetValues GetValues HODM Michael Piasecki Drexel University

CUAHSI HIS Update Current Status of the Project HIS 1.0 Components –WaterOneFlow web services –Observations data model Digital Watershed

Hydrologic Science Hydrologic conditions (Fluxes, flows, concentrations) Hydrologic Process Science (Equations, simulation models, prediction) Hydrologic Information Science (Observations, data models, visualization Hydrologic environment (Dynamic earth) Physical laws and principles (Mass, momentum, energy, chemistry) It is as important to represent hydrologic environments precisely with data as it is to represent hydrologic processes with equations

Data Cube Space, L Time, T Variables, V D “What” “Where” “When” A simple data model

Continuous Space-Time Model – NetCDF (Unidata) Space, L Time, T Variables, V D Coordinate dimensions {X} Variable dimensions {Y}

Space, FeatureID Time, TSDateTime Variables, TSTypeID TSValue Discrete Space-Time Data Model ArcHydro

Terrain Data Models Grid Contour and flowline TIN

CUAHSI Observations Data Model A relational database at the single observation level (atomic model) Stores observation data made at points Metadata for unambiguous interpretation Traceable heritage from raw measurements to usable information Streamflow Flux tower data Precipitation & Climate Groundwater levels Water Quality Soil moisture data

Ernest To Center for Research in Water Resources University of Texas at Austin What are the basic attributes to be associated with each single observation and how can these best be organized? A data source operates an observation network A network is a set of observation sites Data Source and Network SitesVariablesValuesMetadata Depth of snow pack Streamflow Landuse, Vegetation Windspeed, Precipitation Data Delivery Controlled Vocabulary Tables e.g. mg/kg, cfs e.g. depth e.g. Non-detect,Estimated, A site is a point location where one or more variables are measured Metadata provide information about the context of the observation. A variable is a property describing the flow or quality of water A value is an observation of a variable at a particular time Data Discovery Hydrologic Observations Data Model See

1 1 CouplingTable SiteID (GUID) HydroID (Integer) MonitoringPoint SiteID SiteCode SiteName Latitude Longitude … Hydrologic Observations Data Model 1 1 OR Independent of, but coupled to Geographic Representation HODM Arc Hydro

Variable attributes VariableName, e.g. discharge VariableCode, e.g SampleMedium, e.g. water Valuetype, e.g. field observation, laboratory sample IsRegular, e.g. Yes for regular or No for intermittent TimeSupport (averaging interval for observation) DataType, e.g. Continuous, Instantaneous, Categorical GeneralCategory, e.g. Climate, Water Quality NoDataValue, e.g m 3 /s L 3 /T Cubic meters per second

Stage and Streamflow Example

Daily Average Discharge Example Daily Average Discharge Derived from 15 Minute Discharge Data

Water Chemistry from a profile in a lake

ODM and HIS in an Observatory Setting e.g.

WaterOneFlow Web Services Data accessthrough web services Data storage through web services Downloads Uploads Observatory data servers CUAHSI HIS data servers 3 rd party data servers e.g. USGS, NCDC GIS Matlab IDL Splus, R Excel Programming (Fortran, C, VB) Web services interface HDAS Website Portal and Map Viewer Information input, display, query and output services Preliminary data exploration and discovery. See what is available and perform exploratory analyses HTML -XML WSDL - SOAP

CUAHSI HIS Update Current Status of the Project HIS 1.0 Components –WaterOneFlow web services –Observations data model Digital Watershed

How can hydrologists integrate observed and modeled data from various sources into a single description of the environment?

Digital Watershed Hydrologic Observation Data Geospatial Data Weather and Climate Data Remote Sensing Data (NetCDF) (GIS) (Relational database) (EOS-HDF) Digital Watershed A digital watershed is a synthesis of hydrologic observation data, geospatial data, remote sensing data and weather and climate data into a connected database for a hydrologic region

NHDPlus for Region 17E

NHDPlus Reach Catchments ~ 3km 2 About 1000 reach catchments in each 8-digit HUC Average reach length = 2km 2.3 million reaches for continental US

Reach Attributes Slope Elevation Mean annual flow –Corresponding velocity Drainage area % of upstream drainage area in different land uses Stream order

Project sponsored by the European Commission to promote integration of water models within the Water Framework Directive Software standards for model linking Uses model core as an “engine”

OpenMI Conceptual Framework VALUES All values are referenced in a what-where-when framework, allowing different data resources or models to communicate data Space, L Time, T Variables, V D An application of the data cube to integrate simulation models Jon Goodall, Duke University

HIS as OpenMI Components water balance model NWIS Streamflow Daymet Precipitation To calculate storage, the model needs inflow and outflow To calculate storage, the model needs precipitation Goal: Link the National HIS web services with a simple water balance model using OpenMI as the mediator Trigger: Calculate storage

Watershed Hydrovolumes Geovolume is the portion of a hydrovolume that contains solid earth materials USGS Gaging stations Hydrovolume

Stream channel Hydrovolumes Need the capacity to represent Acoustic Doppler Current Profiler (ADCP) data (Iowa) Residence time distributions

Integration of surface water and groundwater data Describe the relationship between surface water features ( e.g. streams and waterbodies) with groundwater features (aquifers, wells). Enable the connection with the surface water data model Hydro networkAquifers In the future go to 3D...

Water OneFlow We need a “Water OneFlow” – a common window for water data and models Advancement of water science is critically dependent on integration of water information Federal Academic Local State Model Flow Precip