1. Hydrologic Modeling in 2011 David R. Maidment Center for Research in Water Resources University of Texas at Austin Leader of the CUAHSI Hydrologic Information System Project With acknowledgements to Rick Hooper, David Tarboton & Barbara Minsker

2. Hydrologic Modeling in 2011 The charge and challenges Hydrologic information system – web services Integrating models and data using scientific workflows Hydrologic Observing System

3. Hydrologic Modeling in 2011 The charge and challenges Hydrologic information system – web services Integrating models and data using scientific workflows Hydrologic Observing System

4. Workshop Charge What new technologies for observing, simulating, and tele-communicating will emerge over the next 5-10 years? how will they change the grand challenges for modeling, what will those challenges be? Challenge for this session: –How all the new devices/opportunities emerging in the realm of “cyber-infrastructure”— including, perhaps especially, visualization schemes — might change the way models are developed and applied, including the new kinds of scientific questions to be asked in association with modeling.

5. Hydrologic Modeling We want to trace the movement of water, chemical and biological constituents through atmospheric, surface and subsurface water We want to do water, mass and energy balances

6. Hydrologic Information System A system is a connected set of components e.g. University of Texas System A web-based system is a set of components connected using the internet A hydrologic information system (HIS) is a web- based system linking hydrologic databases, tools and models CUAHSI HIS partner institutions

7. USGS Water Watch System A national hydrologic observing system already exists – CUAHSI adds to it

8. Real-time Water Quality Estimates Estimated total nitrogen Stream discharge mg/L cfs

9. CUAHSI Member Institutions 105 Universities as of May 2006

10. Challenges How to use test-beds to design real WATERS Observatories? How to share data from the test-beds with the whole community? How to include CUAHSI/CLEANER data not collected in the test-beds? How to empower individual scientists? How to make use of petascale computing?

11. Hydrologic Modeling in 2011 The charge and challenges Hydrologic information system – web services Integrating models and data using scientific workflows Hydrologic Observing System

12. CUAHSI Web Services CUAHSI Web Services Library Web Application: Data Portal Your application Excel, ArcGIS, Matlab Fortran, C/C++, Visual Basic Hydrologic model ……………. Your operating system Windows, Unix, Linux, Mac Internet Simple Object Access Protocol http://www.cuahsi.org/HIS/

13. Rainfall & Snow Water quantity and quality Remote sensing Water Data Modeling Meteorology Soil water

14. Water Data Web Sites

15. NWISWeb site output # agency_cd Agency Code # site_no USGS station number # dv_dt date of daily mean streamflow # dv_va daily mean streamflow value, in cubic-feet per-second # dv_cd daily mean streamflow value qualification code # # Sites in this file include: # USGS 02087500 NEUSE RIVER NEAR CLAYTON, NC # agency_cdsite_nodv_dtdv_vadv_cd USGS020875002003-09-011190 USGS020875002003-09-02649 USGS020875002003-09-03525 USGS020875002003-09-04486 USGS020875002003-09-05733 USGS020875002003-09-06585 USGS020875002003-09-07485 USGS020875002003-09-08463 USGS020875002003-09-09673 USGS020875002003-09-10517 USGS020875002003-09-11454 Time series of streamflow at a gaging station

16. CUAHSI Hydrologic Data Access System A common data window for accessing, viewing and downloading hydrologic information USGS NASANCDC EPANWS Observatories http://river.sdsc.edu/HDAS

17. Observation Stations Ameriflux Towers (NASA & DOE)NOAA Automated Surface Observing System USGS National Water Information SystemNOAA Climate Reference Network Map for the US

18. NWIS Station Observation Metadata Describe what has been measured at this station

19. Web Page Scraping Programmatically construct a URL string as produced by manual use of the web page http://nwis.waterdata.usgs.gov/nwis/discharge?site_no=02087500&agency_cd=USGS&.... Parse the resulting ASCII file

20. NWIS ArcGIS Excel NCAR Unidata NASA Storet CUAHSI Ameriflux Matlab AccessSAS Fortran Visual Basic C/C++ Some operational services CUAHSI Web Services Data Sources Applications Extract Transform Load http://www.cuahsi.org/his/

21. Core Web Methods MethodInputOutput GetSites Obs NetworkAll station codes in network GetSiteInfo Station CodeLat/long, station name GetVariables Obs Network or data source All variable codes GetVariableInfo Variable codeDescription of variable GetValues Station code or lat/long point, variable code, begin date, end date A time series of values GetChart As for GetValueA chart plotting the values

22. Operational Services Service AmerifluxDaymetMODISNWISNAMHODM Bear Creek GetSites Yes GetSiteInfo Yes GetVariables Yes GetVariableInfo Yes GetValues Yes GetChart Yes

23. XML Output from GetValues NWIS DayMet MODIS

24. What is a Data Model A data model is a model that describes in an abstract way how data is represented Data models describe structured data for storage in data management systems such as relational databases. Early phases of many software development projects emphasize the design of a conceptual data model. Lets see what Wikipedia says

25. CUAHSI Point Hydrologic Observations Data Model A relational database stored in Access, PostgreSQL, SQL/Server, …. Stores observation data made at points Consistent format for storage of observations from many different sources and of many different types. Streamflow Flux tower data Precipitation & Climate Groundwater levels Water Quality Soil moisture data

26. Hydrologic Observations Data Model (HODM)

27. Serving investigator data Several choices –You build CUAHSI compatible services from your database –You copy data into the HODM and use CUAHSI services –You copy your data to an HODM and it is served from SDSC Your database Your implementation of CUAHSI services HODM Standard CUAHSI services

28. Modeling Services Simulation models can be packaged as web services They can be queried and provide responses just like data archives We have an integrated network of data sources and models A big challenge to integrate all the data streams!

29. Search multiple heterogeneous data sources simultaneously regardless of semantic or structural differences between them Objective NWIS NARR NAWQA NAM-12 request request return return What we don’t want ….. Michael Piasecki Drexel University

30. Semantic Mediator What we do want ….. NWIS NAWQA NARR generic request request HODM Michael Piasecki Drexel University

31. Hydrologic Modeling in 2011 The charge and challenges Hydrologic information system – web services Integrating models and data using scientific workflows Hydrologic Observing System

32. Regional Storm Water Modeling Program and Master Plan for San Antonio City of San Antonio

33. Modeling System Rainfall Data: Rain gages Nexrad Calibration Data: Flows Water Quality Geospatial Data: City, County SARA, other Floodplain Management Integrated Regional Water Resources planning Capital Improvement Planning Flood Forecasting Water quality planning San Antonio Regional Watershed Modeling System “Bring the models together”

34. Database Geo- HMS Geo- RAS GIS- Gflow Interface HEC-HMS HEC-RAS Gflow GIS GIS Preprocessors for Hydrologic Models

35. Interface data models HMS RAS Gflow GIS Geo Database Arc Hydro data model Connecting Arc Hydro and Hydrologic Models

36. Digital Rain Maps from National Weather Service (03/04/2004)

37. FEMA 100-year flood plain map in Bexar County

38. Regional Watershed Modeling System Case Study Rosillo Creek watershed Arc Hydro Geodatabase for whole watershed HEC-HMS hydrology model for whole watershed HEC-RAS hydraulic model for Rosillo Creek Salado Creek watershed Components: Bexar County

39. Arc Hydro and HEC-HMS Arc Hydro Schematic Network HEC-HMS Hydrologic Model Calculates Flows

40. Arc Hydro and HEC-RAS Arc Hydro Channel Cross Sections HEC-RAS Hydraulic Model Calculates Water Surface Elevations

41. Flow Change Points Models communicate with one another through Arc Hydro at designated points

42. Nexrad Map to Flood Map in Arc 9 Model Builder FLO ODP LAIN MAP Flood map as output Model for flood flow Model for flood depth HMS Nexrad rainfall map as input

43. Web-Accessible Regional Watershed Modeling System Complete storage of simulation models and workflows in geodatabases

44. Hydrologic Modeling in 2011 The charge and challenges Hydrologic information system – web services Integrating models and data using scientific workflows Hydrologic Observing System

45. CUAHSI Hydrologic Observing System Continental US Scale (coast to coast data coverage, HIS-USA) 1:500,000 scale Regional Scale (e.g. Neuse basin) 1:100,000 scale Watershed Scale (e.g. Eno watershed ) 1:24,000 scale Site Scale (experimental site level) Site scale Multiscale information delivery A multiscale web portal system for observing and interpreting hydrologic phenomena by integrating data and models for any location or region in the United States PointPoint Observation Scale (gage, sampling location) North American Scale (e.g. North American Regional Reanalysis of climate) 1:1,000,000 scale

46. GeoTemporal Reference Frame A defined geospatial coordinate system for (x,y,z) A defined time coordinate system (UTC, Eastern Standard Time, ….) A set of variables, V Data values v(x,y,z,t) Space (x,y,z) Time, t Variables, V v – data values Data Cube

47. Series and Fields Features Point, line, area, volume Discrete space representation Series – ordered sequence of numbers Time series – indexed by time Frequency series – indexed by frequency Surfaces Fields – multidimensional arrays Scalar fields – single value at each location Vector fields – magnitude and direction Random fields – probability distribution Continuous space representation

48. mm / 3 hours Precipitation Evaporation North American Regional Reanalysis of Climate Variation during the day, July 2003 NetCDF format

49. Continuous Space-Time Model – NetCDF (Unidata) Space, L Time, T Variables, V D Coordinate dimensions {X} Variable dimensions {Y}

50. Space, FeatureID Time, TSDateTime Variables, TSTypeID TSValue Discrete Space-Time Data Model ArcHydro

51. Hydrologic Flux Coupler Precipitation Evaporation Streamflow Define the fluxes and flows associated with each hydrovolume Groundwater recharge See Chapter 9 of Status Report for Details

52. ArcGIS ModelBuilder Application for Automated Water Balancing Fields Series Geospatial

53. Water Resource Regions and HUC’s

54. NHDPlus for Region 17E

55. NHDPlus Reach Catchments ~ 3km 2

56. Reach Attributes Slope Elevation Mean annual flow –Corresponding velocity Drainage area % of upstream drainage area in different land uses Stream order

57. Ingestion of real-time streamflow data A national hydrologic observing system already exists – CUAHSI adds to it

58. Continental Water Dynamics Model Hydrologic Information System Hydrologic Observing System Hydrologic Modeling System

59. Petascale Computing 2.6 million river reaches on a 1:100,000 scale map of continental US Solve continuity and momentum equations once on each reach (~ 5.2 million equations) takes ~ 200 parallel processors Pittsburgh Supercomputer Center has 3000 parallel processors It is within reach to simulate flows on all reaches continuously through time with data assimilation from gaging stations

60. Conclusions Web services support a web-based hydrologic information system connnecting data, tools and models Models can be configured as web services Scientific workflows automate the integration of components A continental water dynamics model is feasible