GIS in Water Resources David R. Maidment Center for Research in Water Resources University of Texas at Austin GIS Day @ KU 19 November 2008
GIS and Water Resources WaterML – Water Data Language Observations Data Model Observations Data Layers Harvesting water data in GIS
GIS and Water Resources WaterML – Water Data Language Observations Data Model Observations Data Layers Harvesting water data in GIS
GIS and Water Resources Data: Static in time Complex in space Standardized formats Data: Dynamic in time Simple in space (points) No standardized formats
What is “Hydro”? Hydrology Hydrography Properties of Water – WaterML Circulation of the waters of the earth through the hydrologic cycle The “blue lines” on maps Properties of Water – WaterML Features of Water Environment
What is CUAHSI? UCAR CUAHSI – Consortium of Universities for the Advancement of Hydrologic Science, Inc Formed in 2001 as a legal entity Program office in Washington (5 staff) NSF supports CUAHSI to develop infrastructure and services to advance hydrologic science in US universities Unidata Atmospheric Sciences Earth Sciences Ocean Sciences CUAHSI National Science Foundation Geosciences Directorate HIS
CUAHSI Member Institutions 122 Universities as of October 2008
Water quantity and quality Water Data Water quantity and quality Rainfall Soil water Modeling Meteorology Groundwater
Water Data Web Sites
HTML as a Web Language HyperText Text and Pictures Markup Language <title>Texas Water Development Board</title> <!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"> <html> <head> <meta name = "Robots" content = "index,follow"> <meta name = "Priority" content = "home,twdb,homepage"> <meta name = "Author" content = "Texas Water Development Board, Agency Number 580"> <meta name = "Title" content = "Texas Water Development Board"> <meta name = "Description" content = "Texas Water Development Board Home Page"> <meta name = "Keywords" content = "water,drought,rain,conservation,groundwater,surfacewater,lake,reservoir,hydrology,geology,desalination,TWDB,loans,grants,wastewater,sewage,Clean Water,Drinking Water,State Revolving Fund,planning,State Water Plan,GIS,Geographic Information Systems,Mapping,data"> HyperText Markup Language Text and Pictures in Web Browser
WaterML as a Web Language Discharge of the San Marcos River at Luling, June 28 - July 18, 2002 Streamflow data in WaterML language
Point Observations Information Model for USGS Daily Values Data Source Streamflow gages Network GetSites GetSiteInfo San Marcos River at Luling, TX (Site: 08172000) Sites GetVariables Discharge, stage (Daily or instantaneous) Variables GetVariableInfo GetValues Values 19000 cfs, 6 July 2002, A {Value, Time, Qualifier}
WaterML and WaterOneFlow Locations Variable Codes Date Ranges TWDB Data GetSiteInfo GetVariableInfo GetValues Texas A&M NWIS Data WaterML Data WaterOneFlow Web Service Data Repositories Client TRANSFORM EXTRACT LOAD WaterML is an XML language for communicating water data WaterOneFlow is a set of web services based on WaterML
WaterOneFlow Set of query functions Returns data in WaterML
Services-Oriented Architecture for Water Data Links geographically distributed information servers through internet Web Services Description Language (WSDL from W3C) We designed WaterML as a web services language for water data Functions for computer to computer interaction HIS Servers in the WATERS Network HIS Central at San Diego Supercomputer Center Web Services
National Water Metadata Catalog HIS Central Get Data WaterML National Water Metadata Catalog Get Metadata
Data Sources Extract Transform CUAHSI Web Services Load Applications NASA Storet Snotel Extract NCDC Unidata NWIS Academic Transform CUAHSI Web Services Excel Visual Basic Shauna – the ones on top are a bunch of web sites providing data that go through the CUAHSI Web Services (basically some software) which allows all this data to be used in all these other software packages listed across the bottom. The ones in red are the ones that are connections that are already built. It would look good to redo this in ESRI style. ArcGIS Java Load Matlab Applications http://www.cuahsi.org/his/ Operational services
We are at a tipping point …. Web pages Web services Internet Internet Computer Person Computer Computer People interact with a remote information server Networks of information servers provide services to one another
Information communication Water web pages Water web services Water Markup Language (WaterML) HyperText Markup Language (HTML)
GIS and Water Resources WaterML – Water Data Language Observations Data Model Observations Data Layers Harvesting water data in GIS
CUAHSI Point Observation Data Services Data Loading Put data into the CUAHSI Observations Data Model Data Publishing Provide web services access to the data Data Indexing Summarize the data in a centralized cataloging system
CUAHSI Point Observation Data Services Data Loading Put data into the CUAHSI Observations Data Model Data Publishing Provide web services access to the data Data Indexing Summarize the data in a centralized cataloging system
Data Values – indexed by “What-where-when” Time, T “When” t A data value vi (s,t) “Where” s Space, S Vi “What” Variables, V
Data Values Table Time, T t vi (s,t) s Space, S Vi Variables, V
Observations Data Model Horsburgh, J. S., D. G. Tarboton, D. R. Maidment and I. Zaslavsky, (2008), "A Relational Model for Environmental and Water Resources Data," Water Resour. Res., 44: W05406, doi:10.1029/2007WR006392.
HIS Implementation in WATERS Network Information System National Hydrologic Information Server San Diego Supercomputer Center 11 WATERS Network test bed projects 16 ODM instances (some test beds have more than one ODM instance) Data from 1246 sites, of these, 167 sites are operated by WATERS investigators
CUAHSI Point Observation Data Services Data Loading Put data into the CUAHSI Observations Data Model Data Publishing Provide web services access to the data Data Indexing Summarize the data in a centralized cataloging system
ODM Data Loader Publishing an ODM Water Data Service WaterML Texas A&M Corpus Christi Utah State University University of Florida Assemble Data From Different Sources ODM Data Loader Ingest data using ODM Data Loader WaterML Observations Data Model (ODM) Load Newly Formatted Data into ODM Tables in MS SQL/Server USU ODM UFL ODM TAMUCC ODM Wrap ODM with WaterML Web Services for Online Publication
Snotel Web Site in Portland, OR Publishing a Hybrid Water Data Service Snotel Metadata are Transferred to the ODM WaterML Snotel DataValues Snotel METADATA ODM Snotel Water Data Service Web Services can both Query the ODM for Metadata and use a Web Scraper for Data Values Get Values from: Metadata From: ODM Database in San Diego, CA Snotel Web Site in Portland, OR Calling the WSDL Returns Metadata and Data Values as if from the same Database http://river.sdsc.edu/snotel/cuahsi_1_0.asmx?WSDL
CUAHSI Point Observation Data Services Data Loading Put data into the CUAHSI Observations Data Model Data Publishing Provide web services access to the data Data Indexing Summarize the data in a centralized cataloging system
Data Series – Metadata description Space Variable, Vi Site, Sj End Date Time, t2 Begin Date Time, t1 Time Variables Count, C There are C measurements of Variable Vi at Site Sj from time t1 to time t2
Series Catalog Sj Space Variable, Vi Site, Sj End Date Time, t2 Begin Date Time, t1 Time Variables Count, C Vi t1 t2 C
CUAHSI National Water Metadata Catalog Indexes: 50 observation networks 1.75 million sites 8.38 million time series 342 million data values NWIS STORET TCEQ
Searching each data source separately Data Searching Search multiple heterogeneous data sources simultaneously regardless of semantic or structural differences between them Searching each data source separately NWIS return request request return request return NAWQA NAM-12 request return return request return request return request request return NARR Michael Piasecki Drexel University
Searching all data sources collectively Semantic Mediation Searching all data sources collectively GetValues NWIS GetValues GetValues GetValues generic request GetValues NAWQA GetValues NARR Michael Piasecki Drexel University GetValues HODM GetValues
Hydroseek http://www.hydroseek.org Bora Beran, Drexel Supports search by location and type of data across multiple observation networks including NWIS and Storet
HydroTagger Ontology: A hierarchy of concepts Each Variable in your data is connected to a corresponding Concept
National Water Metadata Catalog Synthesis and communication of the nation’s water data http://his.cuahsi.org Government Water Data Academic Water Data National Water Metadata Catalog Hydroseek WaterML
GIS and Water Resources WaterML – Water Data Language Observations Data Model Observations Data Layers Harvesting water data in GIS
Prototype Texas HIS Texas Water Development Board is supporting a project at UT to start building a prototype Texas Hydrologic Information System HIS servers at data sources (State agencies, River authorities, Water Districts, Cities, Counties….) Web Services Texas Hydrologic Information Server (at TNRIS) Texas Observations Catalogs and some state water datasets
Levels of Government National data services (USGS, EPA, NCDC, NWS...) Web State data services (TCEQ, TWDB, TCEQ, ….) Services Regional data services (LCRA, BRA, City of Austin, ...)
Texas Hydrologic Information System Sponsored by the Texas Water Development Board and TNRIS using CUAHSI technology for state and local data sources (using state funding)
Observations Data Layer for Water Quality in Texas Attributes are time series of: Bacterial concentrations Water temperature Nitrogen components …….
GIS and Water Resources WaterML – Water Data Language Observations Data Model Observations Data Layers Harvesting water data in GIS
What is HydroGET? A web service client for ESRI‘s ArcGIS environment. Harvests time series data from data repositories on the web and stores them in the ArcHydro data model. Default Mode: Downloads data from national data sources to describe components of the hydrological cycle. Atmospheric data from Daymet or Unidata Surface data from USGS NWIS Subsurface data from USGS NWIS Illustration courtesy of the United States Geological Survey Custom Mode: Downloads data from any combination of CUAHSI web services to describe other properties (e.g. biological) of a study area.
HydroGet
HydroGET Interface Default Mode Custom Mode Types of data and sources User inputs Main Interface GIS layer that contains points of interest. Atmospheric data from Daymet and Unidata Default Mode Target geodatabase for downloaded data Surface data from USGS NWIS List of variables of interest (each tab holds a different set of variables) Subsurface data from USGS NWIS Data from user- specified sources for single point Custom Mode Period of interest Data from user- specified sources for multipoints
Assembling data for hydrological insights Precipitation data from watershed centroids Streamflow data from USGS gages. Groundwater data from USGS wells. Precipitation (cm) Streamflow (cfs) Groundwater level (m above ground surface)
What is MySelect? A featureclass with an attribute table that contains query parameters for downloading and storing web service data. Each record in the table is one web service request. A MySelect table can have several requests, essentially making it a shopping list for environmental data. Get precipitation Get streamflow Get groundwater level
MySelect in action Map of Corpus Christi Bay in southeast Texas ! > LEGEND TCOON (Texas Coastal Ocean Observation Network ) platforms ! > > ! HRI (Harte Research Institute ) Stations SERF (Shoreline Environmental Research Facility) platforms > ! Plume tracking stations (Dr. Ben Hodges) This demonstration will show the use of MySelect in getting data from regional data sets (i.e. wind from TCOON and salinity from HRI). Evidence of hypoxic conditions Map of Corpus Christi Bay in southeast Texas
Semantic mediation with MySelect TSTypeID/VarID Variable 1001 Dissolved_Oxygen_Concentration 1002 Salinity 1003 Temperature 1005 Wind_Direction 1006 Wind_Speed HRI: DOConcCon HRI: DOConGrab HRI: DOBottomGrab HRI: DOBottomCon Hodges: DO MySelect SERF: oxygen HRI: SalinityCon HRI: SalinityGrab Hodges: Salinity SERF: salinity HRI: TempGrab HRI: TempCon Hodges: Temperature SERF: temperature TCOON: wdr TCOON: wsd Data sources may have different names for the same variable. However, differences can be sorted out in ArcHydro by appropriately assigning related variables with the same TSTypeID/ VarID.
Example: DataCube diagrams Subset table with x,y,z,t coordinates and oxygen values DataCube diagrams
Conclusions A new web services technology has emerged for water observations data From GIS perspective, this means the creation of observations data layers of water data time series Texas is building a Hydrologic Information System using the CUAHSI approach Could Kansas do the same? For more information, see http://his.cuahsi.org