1. Waters Observatory Network Observe water systems Observe in many different contexts and for different purposes Have a Waters Network Information System for sharing data –Common language for data –Geographic federation of dispersed data sources through web services

2. Project co-PI in Phase 2 Collaborator in Phase I CUAHSI HIS Partner Institutions

3. 10 WATERS Testbed Sites

4. HIS Team WATERS Testbed WATERS Network Information System

5. Super Computer Centers: NCSA, TACC Domain Sciences: Unidata, NCAR LTER, GEON Government: USGS, EPA, NCDC, USDA Industry: ESRI, Kisters, Microsoft CUAHSI HIS HIS Team and its Cyberinfrastructure Partners HIS Team: Texas, SDSC, Utah, Drexel, Duke

6. Super Computer Centers: NCSA TACC Domain Sciences: Unidata, NCAR LTER, GEON Government: USGS, EPA, NCDC, USDA Industry: ESRI, Kisters, Microsoft HIS Team WATERS Testbed WATERS Network Information System CUAHSI HIS HIS and WATERS

7. CUAHSI Member Institutions 116 Universities as of November 2006

8. Super computer Centers: NCSA, TACC Domain Sciences: Unidata, NCAR LTER, GEON Government: USGS, EPA, NCDC, USDA Industry: ESRI, Kisters, OpenMI HIS Team WATERS Testbed WATERS Network Information System CUAHSI HIS HIS, WATERS and the CUAHSI Community

9. KISTERS Water Resources Management Data System KISTERS is building CUAHSI web services onto their database Stefan Fuest

10. Super computer Centers: NCSA, TACC Domain Sciences: Unidata, NCAR LTER, GEON Government: USGS, EPA, NCDC, USDA Industry: ESRI, Kisters, OpenMI HIS Team WATERS Testbed WATERS Network Information System CUAHSI HIS International Partners CSIRO Land and Water Resources Water Resources Observations Network (WRON) European Commission Water database design and model integration (HarmonIT and OpenMI)

13. HIS Goals Hydrologic Data Access System – better access to a large volume of high quality hydrologic data Support for Observatories – synthesizing hydrologic data for a region Advancement of Hydrologic Science – data modeling and advanced analysis Hydrologic Education – better data in the classroom, basin-focused teaching

14. Rainfall & Snow Water quantity and quality Remote sensing Water Data Modeling Meteorology Soil water

15. Water Data Web Sites

16. NWISWeb site output # agency_cd Agency Code # site_no USGS station number # dv_dt date of daily mean streamflow # dv_va daily mean streamflow value, in cubic-feet per-second # dv_cd daily mean streamflow value qualification code # # Sites in this file include: # USGS 02087500 NEUSE RIVER NEAR CLAYTON, NC # agency_cdsite_nodv_dtdv_vadv_cd USGS020875002003-09-011190 USGS020875002003-09-02649 USGS020875002003-09-03525 USGS020875002003-09-04486 USGS020875002003-09-05733 USGS020875002003-09-06585 USGS020875002003-09-07485 USGS020875002003-09-08463 USGS020875002003-09-09673 USGS020875002003-09-10517 USGS020875002003-09-11454 Time series of streamflow at a gaging station USGS has committed to supporting CUAHSI’s GetValues function

17. Observation Stations Ameriflux Towers (NASA & DOE)NOAA Automated Surface Observing System USGS National Water Information SystemNOAA Climate Reference Network Map for the US

18. Water Quality Measurement Sites in EPA Storet Substantial variation in data availability from states Data from Bora Beran, Drexel University

19. Water Quality Measurement Sites from Texas Commission for Environmental Quality (TCEQ)

20. Geographic Integration of Storet and TCEQ Data in HIS

21. Observations Catalog Specifies what variables are measured at each site, over what time interval, and how many observations of each variable are available

22. Point Observations Information Model Data Source Network Sites Variables Values {Value, Time, Qualifier} USGS Streamflow gages Neuse River near Clayton, NC Discharge, stage (Daily or instantaneous) 206 cfs, 13 August 2006 A data source operates an observation network A network is a set of observation sites A site is a point location where one or more variables are measured A variable is a property describing the flow or quality of water A value is an observation of a variable at a particular time A qualifier is a symbol that provides additional information about the value

23. Data Discovery and Delivery Data Source Network Sites Variables Values Observations metadata Observations data HIS Server Observations Catalog Web services Data Discovery Data Delivery HIS facilitates data discovery by building and maintaining observations catalogs Data delivery occurs through web services from remote data archives or local observations databases. Water resource agencies support data delivery services.

24. NWIS ArcGIS Excel NCAR Unidata NASA Storet NCDC Ameriflux Matlab AccessJava Fortran Visual Basic C/C++ Some operational services CUAHSI Web Services Data Sources Applications Extract Transform Load http://www.cuahsi.org/his/

25. WaterOneFlow Web Services Data accessthrough web services Data storage through web services Downloads Uploads Observatory servers Workgroup HIS SDSC HIS servers 3 rd party servers e.g. USGS, NCDC GIS Matlab IDL Splus, R D2K, I2K Programming (Fortran, C, VB) Web services interface Web portal Interface (HDAS) Information input, display, query and output services Preliminary data exploration and discovery. See what is available and perform exploratory analyses HTML -XML WSDL - SOAP Hydrologic Information System Service Oriented ArchitectureNWIS ArcGIS Excel NCAR Unidata NASA Storet NCDC Ameriflux Matlab AccessSAS Fortran Visual Basic C/C++ CUAHSI Web Services Ilya Zaslavsky

26. My server Web services: key terms Web Service Code WSDL Web service description language Your Web application Reference a web service, and call a WS method Your desktop application Reference a web service, and call a WS method SOAP Messages SOAP: Simple Object Access Protocol

27. - From different database structures, data collection procedures, quality control, access mechanisms  to uniform signatures … Water Markup Language - Tested in different environments - Standards-based - Can support advanced interfaces via harvested catalogs - Accessible to community - Templates for development of new services - Optimized, error handling, memory management, versioning, run from fast servers And: working with agencies on setting up services!

28. Web service testing pages http://water.sdsc.edu/webservicetestpage/

29. For Developers and Information Managers Code developers –HIS Team System: code repository, web services –Design Documents –Workbook for using web services in different environments –Web service testing and versioning guidelines and modules –Support Site managers –Step-by-step instructions for setting up the server (eventually, will be an “HIS appliance”) –Configuration and data management tools What it takes to setup and maintain HIS server, and how to get there from your current setup – THIS AFTERNOON

30. CUAHSI Hydrologic Information System Levels National HIS – San Diego Supercomputer Center Workgroup HIS – research center or academic department Personal HIS – an individual hydrologic scientist HIS Server HIS Analyst Map interface, observations catalogs and web services for national data sources Map interface, observations catalogs and web services for regional data sources; observations databases and web services for individual investigator data Application templates and HydroObjects for direct ingestion of data into analysis environments: Excel, ArcGIS, Matlab, programming languages; MyDB for storage of analysis data

31. Data Ingestion MatlabExcelArcGISJava.Net HydroObjects HIS Analyst HIS Server WaterOneFlow Services Observations Data Tim Whiteaker

32. Tutorials and Templates that show how to use Web services in various applications ftp://ftp.crwr.utexas.edu/pub/outgoing/CUAHSI/HIS_workbook/20061115/ Includes HydroObjects Library

33. HIS Analyst Demo (Tim Whiteaker) Ingestion of data using web services into Excel and VB.Net

34. CUAHSI Hydrologic Data Access System A common data window for accessing, viewing and downloading hydrologic information USGS NASANCDC EPANWS Observatory Data http://river.sdsc.edu/HDAS

35. HIS Server Supports data discovery, delivery and publication –Data discovery – how do I find the data I want? Map interface and observations catalogs –Data delivery – how do I acquire the data I want? Use web services or retrieve from local database –Data Publication – how do I publish my observation data? Use Observations Data Model

36. HIS Server Architecture Map front end – ArcGIS Server 9.2 (being programmed by ESRI Water Resources for CUAHSI – led by Dean Djokic, an IHE graduate) Relational database – SQL/Server 2005 or Express Web services library – VB.Net programs accessed as a Web Service Description Language (WSDL)

37. HIS Server Demo (Dean Djokic) ESRI Water Resources Applications Group, Redlands CA (a voluntary contribution to CUAHSI)

38. Digital Watershed How can hydrologists integrate observed and modeled data from various sources into a single description of the environment?

39. Digital Watershed Hydrologic Observation Data Geospatial Data Weather and Climate Data Remote Sensing Data (NetCDF) (GIS) (Relational database) (EOS-HDF) Digital Watershed A digital watershed is a synthesis of hydrologic observation data, geospatial data, remote sensing data and weather and climate data into a connected database for a hydrologic region

40. HIS Servers Hydrologic Observations Server GIS Data Server Weather and Climate Server Remote Sensing Server Digital Watershed HIS Servers provide hydrologic observations, weather and climate, GIS and remote sensing data. For HIS version 1.0, the focus is a hydrologic observations server for data from gages and monitoring sites at point locations.

41. Water Resource Regions and HUC’s

42. NHDPlus for Region 17E

43. NHDPlus Reach Catchments ~ 3km 2 About 1000 reach catchments in each 8-digit HUC Average reach length = 2km 2.3 million reaches for continental US

44. National Land Cover Dataset http://landcover.usgs.gov/nationallandcover.html http://seamless.usgs.gov/Get the data:

45. Reach Attributes Slope Elevation Mean annual flow –Corresponding velocity Drainage area % of upstream drainage area in different land uses Stream order

46. http://www.ncgc.nrcs.usda.gov/products/datasets/statsgo/ 1:250,000 Scale Soil Information

47. Hydrologic Landscape Regions USGS Characterization of nature of soil and groundwater systems http://water.usgs.gov/lookup/getspatial?hlrus

48. Arc Hydro: GIS for Water Resources Arc Hydro –An ArcGIS data model for water resources –Arc Hydro toolset for implementation –Framework for linking hydrologic simulation models The Arc Hydro data model and application tools are in the public domain Published in 2002

49. Arc Hydro Links Space and Time Integrate geospatial data for land and water systems You store this information in a relational database or get it from web services Link space and time with HydroID All Hydrofeatures of a testbed area will have a unique HydroID Land base from NHDPlus + added data layers

50. Open Hydro adopts the Arc Hydro data model and exposes it using open source GIS and database tools. Open Hydro also includes a suite of tools for working with hydrologic data and managing the associated hydro geodatabase. Dan Ames, Idaho State University

51. Based on MapWindow GIS: Free open source GIS alternative for data visualization and analysis. MapWindow GIS follows OGC standards and works with ESRI data formats. MapWindow GIS is extensible using VB.NET, VB6, C#, and ActiveX. Sponsored by NASA, NOAA, DOE, and EPA (new platform for BASINS). Nearly 30,000 downloaders so far at: www.MapWindow.org www.MapWindow.org

52. Open Hydro is still under development and ISU is seeking partners to work together on various functions. Current completed functions include geodatabase import/export, and automatic watershed delineation and support for ESRI’s geography network and online data sources.

53. http://www.daymet.org/

54. Demo of NHDPlus and Weather Downloader for the San Marcos Basin By Ernest To Center for Research in Water Resources

55. Search multiple heterogeneous data sources simultaneously regardless of semantic or structural differences between them Objective NWIS NARR NAWQA NAM-12 request request return return What we are doing now ….. Michael Piasecki Drexel University

56. Semantic Mediator What we would like to do ….. NWIS NAWQA NARR generic request GetValues GetValues HODM Michael Piasecki Drexel University

57. Demo on Semantic Mediator By Bora Beran and Michael Piasecki, Duke University

58. Project sponsored by the European Commission to promote integration of water models within the Water Framework Directive Software standards for model linking Uses model core as an “engine” http://www.openMI.org

59. OpenMI – Links Data and Simulation Models CUAHSI Observations Data Model as an OpenMI component Simple River Model Trigger (identifies what value should be calculated)

60. Typical model architecture Application User interface + engine Engine Simulates a process – flow in a channel Accepts input Provides output Model An engine set up to represent a particular location e.g. a reach of the Thames Engine Output data Input data Model application Run Write Read User interface

61. AcceptsProvides Rainfall (mm) Runoff (m 3 /s) Temperature (Deg C) Evaporation (mm) AcceptsProvides Upstream Inflow (m 3 /s) Outflow (m 3 /s) Lateral inflow (m 3 /s) Abstractions (m 3 /s) Discharges (m 3 /s) River Model Linking modelled quantities

62. Data transfer at run time Rainfall runoff Output data Input data User interface River Output data Input data User interface GetValues(..)

63. Models for the processes River (InfoWorks RS) Rainfall (database) Sewer (Mouse) RR (Sobek-Rainfall -Runoff)

64. Data exchange 3 Rainfall.GetValues River (InfoWorks-RS) Rainfall (database) Sewer (Mouse) 2 RR.GetValues 7 RR.GetValues RR (Sobek-Rainfall -Runoff) 1 Trigger.GetValues 6 Sewer.GetValues call data 4 5 8 9

65. OpenMI OpenMI = Open Modeling Interface It is a Standard for Model Linking It is part of HarmonIT, a European Commission funded research project with 14 project partners (universities & software companies) www.openmi.org www.harmonit.org

66. OpenMI Conceptual Framework VALUES All values are referenced in a what-where-when framework, allowing different data resources or models to communicate data

67. HIS as OpenMI Components water balance model NWIS Streamflow Daymet Precipitation To calculate storage, the model needs inflow and outflow To calculate storage, the model needs precipitation Goal: Link the National HIS web services with a simple water balance model using OpenMI as the mediator Trigger: Calculate storage

68. Demo on Intregating Web Services and OpenMI to make a water balance model By Jon Goodall, Duke University

69. Watershed Hydrovolumes Geovolume is the portion of a hydrovolume that contains solid earth materials USGS Gaging stations Hydrovolume

70. Stream channel Hydrovolumes Need the capacity to represent Acoustic Doppler Current Profiler (ADCP) data (Iowa) Residence time distributions (Minnesota)

71. Integration of surface water and groundwater data Describe the relationship between surface water features ( e.g. streams and waterbodies) with groundwater features (aquifers, wells). Enable the connection with the surface water data model Hydro networkAquifers In the future go to 3D...

72. Water OneFlow Like Geospatial OneStop, we need a “Water OneFlow” – a common window for water data and models Advancement of water science is critically dependent on integration of water information Federal Academic Local State