1. A Services Oriented Architecture for Water Resources Data David R. Maidment Center for Research in Water Resources University of Texas at Austin

2. A Services Oriented Architecture for Water Resources Data CUAHSI and WATERS WaterML and WaterOneFlow Hydrologic Information Server Modeling services

3. A Services Oriented Architecture for Water Resources Data CUAHSI and WATERS WaterML and WaterOneFlow Hydrologic Information Server Modeling services

4. Ocean Sciences What is CUAHSI? CUAHSI – Consortium of Universities for the Advancement of Hydrologic Science, Inc Formed in 2001 as a legal entity Program office in Washington (5 staff) NSF supports CUAHSI to develop infrastructure and services to advance hydrologic science in US universities Earth Sciences Atmospheric Sciences UCAR CUAHSI Unidata HIS National Science Foundation Geosciences Directorate

5. CUAHSI Member Institutions 112 US Universities as of September 2007

6. HIS Team WATERS Testbed WATERS Network Information System NSF has funded work at 11 testbed sites, each with its own science agenda. HIS supplies the common information system Minnehaha Creek

7. Super Computer Centers: NCSA, TACC Domain Sciences: Unidata, NCAR LTER, CZEN GEON Government: USGS, EPA, NCDC Industry: ESRI, Kisters, Microsoft CUAHSI HIS HIS Team and its Cyberinfrastructure Partners HIS Team: Texas, SDSC, Utah, Drexel, Duke

8. Super computer Centers: NCSA, TACC Domain Sciences: Unidata, NCAR LTER, GEON Government: USGS, EPA, NCDC Industry: ESRI, Kisters, Microsoft HIS Team WATERS Testbed WATERS Network Information System CUAHSI HIS HIS, WATERS and the CUAHSI Community

9. Super computer Centers: NCSA, TACC Domain Sciences: Unidata, NCAR LTER, GEON Government: USGS, EPA, NCDC, USDA Industry: ESRI, Kisters, OpenMI HIS Team WATERS Testbed WATERS Network Information System CUAHSI HIS International Partners CSIRO and Bureau of Meteorology Australian Water Resources Information System European Commission Water database design and model integration (OpenMI)

10. HIS Goals Hydrologic Data Access – providing better access to a large volume of high quality hydrologic data across the nation Support for Observatories – integrating local observations by academic investigators with hydrologic data for a region Advancement of Hydrologic Science – modeling and analysis of “hydrology in a dynamic earth” Hydrologic Education – bringing more data into the classroom

11. A Services Oriented Architecture for Water Resources Data CUAHSI and WATERS WaterML and WaterOneFlow Hydrologic Information Server Modeling services

12. Definition The CUAHSI Hydrologic Information System (HIS) is a geographically distributed network of data sources and functions that are integrated using web services so that they operate as a connected whole.

13. Services Oriented Architecture Service-oriented Architecture (SOA) is an architectural design pattern that concerns itself with defining loosely-coupled relationships between producers and consumers. A major focus of Web services is to make functional building blocks accessible over standard Internet protocols that are independent from platforms and programming languages. The Web Services Description Language (WSDL, pronounced 'wiz-dəl' or spelled out, 'W- S-D-L') is an XML-based language that provides a model for describing Web services.XMLWeb services (from Wikipedia) Defined by the World Wide Web Consortium (W3C)

14. Web Pages and Web Services http://www.safl.umn.edu/ http://his.safl.umn.edu/SAFLMC/cuahsi_1_0.asmx? Uses Hypertext Markup Language (HTML) Uses WaterML (an eXtended Markup Language for water data)

15. Rainfall & Snow Water quantity and quality Remote sensing Water Data Modeling Meteorology Soil water

16. Water Data Web Sites

17. NWISWeb site output # agency_cd Agency Code # site_no USGS station number # dv_dt date of daily mean streamflow # dv_va daily mean streamflow value, in cubic-feet per-second # dv_cd daily mean streamflow value qualification code # # Sites in this file include: # USGS 02087500 NEUSE RIVER NEAR CLAYTON, NC # agency_cdsite_nodv_dtdv_vadv_cd USGS020875002003-09-011190 USGS020875002003-09-02649 USGS020875002003-09-03525 USGS020875002003-09-04486 USGS020875002003-09-05733 USGS020875002003-09-06585 USGS020875002003-09-07485 USGS020875002003-09-08463 USGS020875002003-09-09673 USGS020875002003-09-10517 USGS020875002003-09-11454 Time series of streamflow at a gaging station USGS has committed to supporting CUAHSI’s GetValues function

18. Observation Stations Ameriflux Towers (NASA & DOE)NOAA Automated Surface Observing System USGS National Water Information SystemNOAA Climate Reference Network Map for the US

19. Water Quality Measurement Sites in EPA Storet Substantial variation in data availability from states Data from Bora Beran, Drexel University

20. Water Quality Measurement Sites from Texas Commission for Environmental Quality (TCEQ)

21. Geographic Integration of Storet and TCEQ Data in HIS

22. Observations Catalog Specifies what variables are measured at each site, over what time interval, and how many observations of each variable are available

23. Point Observations Information Model Data Source Network Sites Variables Values {Value, Time, Qualifier, Offset} USGS Streamflow gages Neuse River near Clayton, NC Discharge, stage (Daily or instantaneous) 206 cfs, 13 August 2006 A data source operates an observation network A network is a set of observation sites A site is a point location where one or more variables are measured A variable is a property describing the flow or quality of water A value is an observation of a variable at a particular time A qualifier is a symbol that provides additional information about the value An offset allows specification of measurements at various depths in water http://www.cuahsi.org/his/webservices.html GetSites GetSiteInfo GetVariables GetVariableInfo GetValues

24. Locations Variable Codes Date Ranges WaterML and WaterOneFlow GetSiteInfo GetVariableInfo GetValues WaterOneFlow Web Service Client STORET NAM NWIS Data Repositories Data EXTRACT TRANSFORM LOAD WaterML WaterML is an XML language for communicating water data WaterOneFlow is a set of web services based on WaterML

25. WaterOneFlow Set of query functions Returns data in WaterML

26. NWIS ArcGIS Excel NCAR Unidata NASA Storet NCDC Ameriflux Matlab AccessJava Fortran Visual Basic C/C++ Some operational services CUAHSI Web Services Data Sources Applications Extract Transform Load http://www.cuahsi.org/his/

27. GetValues for Dissolved Oxygen at Minnehaha Creek http://his.safl.umn.edu/SAFLMC/cuahsi_1_0.asmx?WSDL

28. A Services Oriented Architecture for Water Resources Data CUAHSI and WATERS WaterML and WaterOneFlow Hydrologic Information Server Modeling services

29. Hydrologic Information Server Supports data discovery, delivery and publication –Data discovery – how do I find the data I want? Map interface and observations catalogs Metadata based Search –Data delivery – how do I acquire the data I want? Use web services or retrieve from local database –Data Publication – how do I publish my observation data? Use Observations Data Model

30. Water Resource Regions and HUC’s

31. National Hydrography Dataset

32. NHDPlus for Region 17E

33. NHDPlus Reach Catchments ~ 3km 2 About 1000 reach catchments in each 8-digit HUC Average reach length = 2km 2.3 million reaches for continental US

34. Reach Attributes Slope Elevation Mean annual flow –Corresponding velocity Drainage area % of upstream drainage area in different land uses Stream order

35. Observations Catalog Specifies what variables are measured at each site, over what time interval, and how many observations of each variable are available

36. Hydrologic Information Server Microsoft SQLServer Relational Database Observations Data Geospatial Data GetSites GetSiteInfo GetVariables GetVariableInfo GetValues DASH – data access system for hydrologyWaterOneFlow services ArcGIS Server

37. Minnehaha Creek Experimental Site http://his.safl.umn.edu/DASH/

38. Water Temperature (°C) Dissolved Oxygen (mg/L) Depth (m) Dissolved Oxygen (% saturation)

39. Data Heterogeneity Syntactic mediation –Heterogeneity of format –Use WaterML to get data into the same format Semantic mediation –Heterogeneity of meaning –Each water data source uses its own vocabulary –Match these up with a common controlled vocabulary –Make standard scientific data queries and have these automatically translated into specific queries on each data source

40. Search multiple heterogeneous data sources simultaneously regardless of semantic or structural differences between them Objective NWIS NARR NAWQA NAM-12 request request return return What we are doing now ….. Michael Piasecki Drexel University

41. Semantic Mediator What we would like to do ….. NWIS NAWQA NARR generic request GetValues GetValues HODM Michael Piasecki Drexel University

42. Hydroseek http://www.hydroseek.org http://www.hydroseek.org Supports search by location and type of data across multiple observation networks including NWIS and Storet

43. A Services Oriented Architecture for Water Resources Data CUAHSI and WATERS WaterML and WaterOneFlow Hydrologic Information Server Modeling services

44. Project sponsored by the European Commission to promote integration of water models within the Water Framework Directive Software standards for model linking Uses model core as an “engine” http://www.openMI.org

45. OpenMI Conceptual Framework VALUES All values are referenced in a what-where-when framework, allowing different data resources or models to communicate data Space, L Time, T Variables, V D An application of the data cube to integrate simulation models Jon Goodall, Duke University

46. Typical model architecture Application User interface + engine Engine Simulates a process – flow in a channel Accepts input Provides output Model An engine set up to represent a particular location e.g. a reach of the Thames Engine Output data Input data Model application Run Write Read User interface

47. AcceptsProvides Rainfall (mm) Runoff (m 3 /s) Temperature (Deg C) Evaporation (mm) AcceptsProvides Upstream Inflow (m 3 /s) Outflow (m 3 /s) Lateral inflow (m 3 /s) Abstractions (m 3 /s) Discharges (m 3 /s) River Model Linking modelled quantities

48. Data transfer at run time Rainfall runoff Output data Input data User interface River Output data Input data User interface GetValues(..)

49. Models for the processes River (InfoWorks RS) Rainfall (database) Sewer (Mouse) RR (Sobek-Rainfall -Runoff)

50. Data exchange 3 Rainfall.GetValues River (InfoWorks-RS) Rainfall (database) Sewer (Mouse) 2 RR.GetValues 7 RR.GetValues RR (Sobek-Rainfall -Runoff) 1 Trigger.GetValues 6 Sewer.GetValues call data 4 5 8 9

51. Coupling the HIS with Models using OpenMI ODM Observations Data Model WaterOneFlow Web Services Water Markup Language WOFWaterML MODFLOW HEC-RAS Others OpenMI SWAT HSFP “academic models”

52. Linking the HIS with HEC-RAS via OpenMI HIS WaterOneFlow Web Service for NWIS Realtime Streamflow A HEC-RAS Model Workflow “trigger” Using real-time streamflow from the HIS as a boundary condition for a HEC-RAS simulation.

53. SOAP-based Web Service for Models HEC-RAS USGS NWIS WSDL Simulation Model Database OpenMI Workflow In an effort to build cyberinfrastructure for the hydrologic sciences, we are extending OpenMI to utilize models as web services.

54. Extending OpenMI for Distributed Computing Connects to remote database via web services Connects to remote model via web services Goal: To allow a modeler to create a workflow from OpenMI components that wrap web services. Model linkage designed on client machine

55. Conclusion The CUAHSI Hydrologic Information System (HIS) is a geographically distributed network of hydrologic data sources and functions that are integrated using web services so that they function as a connected whole. For more information: http://www.cuahsi.org/his.htmlhttp://www.cuahsi.org/his.html