CUAHSI and CLEANER Overview: Infrastructure and Informatics Jon Duncan Program Manager

2 Collaborative Large Scale Engineering Analysis Network for Environmental Research (CLEANER) Project Office funded, based in Arlington, VA Project Office funded, based in Arlington, VA Science, Cyberinfrastructure, Social Science, Sensors, Organization committees Science, Cyberinfrastructure, Social Science, Sensors, Organization committees Major infrastructure planning process and more Major infrastructure planning process and more

3 Consortium of Universities for the Advancement of Hydrologic Science, Inc. (CUAHSI) A consortium of 105 research universities, 6 affiliate members, and 8 int’l affiliates A consortium of 105 research universities, 6 affiliate members, and 8 int’l affiliates Incorporated June, 2001 as a non-profit corporation in Washington, DC Incorporated June, 2001 as a non-profit corporation in Washington, DC

4 What is Hydrologic Science? Expands beyond traditional hydrology Expands beyond traditional hydrology Focus on “why” the earth works as it does, like other earth sciences, moving beyond traditional problem-solving orientation Focus on “why” the earth works as it does, like other earth sciences, moving beyond traditional problem-solving orientation Embraces parts of hydrology, geomorphology, hydrogeology, biogeochemistry, … Embraces parts of hydrology, geomorphology, hydrogeology, biogeochemistry, … Hydrologic cycle is central organizing principle Hydrologic cycle is central organizing principle

5 Purpose Science Objective: To further predictive understanding of the terrestrial hydrologic cycle and its linkages with climate and biogeochemical cycles Science Objective: To further predictive understanding of the terrestrial hydrologic cycle and its linkages with climate and biogeochemical cycles Societal Need: Will there be enough water for the next century? Societal Need: Will there be enough water for the next century? …of appropriate quality …of appropriate quality …to meet society’s needs …to meet society’s needs …to maintain the integrity of our ecosystems …to maintain the integrity of our ecosystems

6 Need for CUAHSI Larger-scale, longer-term research to advance science Larger-scale, longer-term research to advance science Enable research at disciplinary boundaries Enable research at disciplinary boundaries Support of larger research teams Support of larger research teams Improve efficiency and effectiveness of data collection and dissemination Improve efficiency and effectiveness of data collection and dissemination

7 What CUAHSI is doing? Hydrologic Information Systems Hydrologic Information Systems Observatories Observatories Hydrologic Measurement Facilities Hydrologic Measurement Facilities Education and Outreach Education and Outreach Hydrologic Synthesis Center Hydrologic Synthesis Center

8 -Mathematical Formulae -Solution Techniques Abstractions in Modeling Physical World Conceptual Frameworks Data Representation Model Representations “Digital Observatory”Real World Measurements Theory/Process Knowledge Perceptions of this place Intuition Water quantity and quality Meteorology Remote sensing Geographically Referenced Mapping Validation DNA Sequences Vegetation Survey Hydrologist Q, Gradient, Roughness? Groundwater Contribution? Snowmelt Processes? Biogeochemist Hyporheic exchange? Mineralogy? Chemistry? Redox Zones? DOC Quality? Geomorphologist Glaciated Valley Perifluvial Well sorted? Thalweg? Aquatic Ecologist Backwater habitat Substrate Size, Stability? Benthic Community Oligotrophic? Carbon source?

9 Data Representation Four-dimensional {x,y,z,t} Continental scope Multi-scale, multi- resolution Points, coverages, dynamic fields Space, L Time, T Variable, V D A data value 1:1200 scale United States 1:500,000 scale River Basin 1:100,000 scale Watershed 1:24,000 scale River reach Point scale A plot 1:1,000,000 scale North American and Global

10 Digital Continent Integrating monitoring and research data yields a single body of information for the country Integrating monitoring and research data yields a single body of information for the country Observatories contribute intensive information to this body Observatories contribute intensive information to this body Observatories are placed within context of climate, geology, soils, etc. but are not assumed to be representative of an area. Observatories are placed within context of climate, geology, soils, etc. but are not assumed to be representative of an area. Digital observatories may be watersheds, aquifers, river reaches, or any region that is part of the continent Digital observatories may be watersheds, aquifers, river reaches, or any region that is part of the continent

11 Observation Stations Ameriflux Towers (NASA & DOE)NOAA Automated Surface Observing System USGS National Water Information SystemNOAA Climate Reference Network Map for the US

12 Inference Space “Transcending place” means testing hypotheses in areas thought to be similar (in some attributes). “Transcending place” means testing hypotheses in areas thought to be similar (in some attributes). Digital continent will enable identification of “similar” areas and (some) data about that spot. Digital continent will enable identification of “similar” areas and (some) data about that spot. Observatories will enable inference about similar regions (e.g., presumably one can infer more about Delaware R. from Potomac than about Rio Grande). Observatories will enable inference about similar regions (e.g., presumably one can infer more about Delaware R. from Potomac than about Rio Grande).

13 DOs are the foundation of EOs Collaboration of Mission and Science Agencies Collaboration of Mission and Science Agencies DO contains both monitoring and research data DO contains both monitoring and research data DO supports hypothesis test, decision support systems, mgmt models DO supports hypothesis test, decision support systems, mgmt models Interdisciplinary communication Interdisciplinary communication Scientists can access multiple conceptualizations to improve understanding Scientists can access multiple conceptualizations to improve understanding Everyone benefits from context provided Everyone benefits from context provided Incentives must exist for people and agencies to want to contribute (and they do!) Incentives must exist for people and agencies to want to contribute (and they do!)

Water Resource Regions and HUC’s

15 NHDPlus for Region 17E

16 NHDPlus Reach Catchments ~ 3km 2 About 1000 reach catchments in each 8-digit HUC Average reach length = 2km2.3 million reaches for continental US

17 Reach Attributes Slope Slope Elevation Elevation Mean annual flow Mean annual flow Corresponding velocity Corresponding velocity Drainage area Drainage area % of upstream drainage area in different land uses % of upstream drainage area in different land uses Stream order Stream order

18 HIS 1.0 (1 Nov 2006) Point Time Series Point Time Series Discovery and Publication Discovery and Publication Agencies Agencies USGS NWIS USGS NWIS NCDC NCDC EPA Storet EPA Storet [LTER Trends] [LTER Trends] Static Federation to Observatory Test Beds Static Federation to Observatory Test Beds

19 CUAHSI Hydrologic Information System Architecture National HIS – San Diego Supercomputer Center Workgroup HIS – river authority, research centre or univ. Personal HIS – an individual scientist or manager HIS Server HIS Analyst Map interface, observations catalogs and web services for national data sources Map interface, observations catalogs and web services for regional data sources; observations databases and web services for individual investigator data Application templates and HydroObjects for direct ingestion of data into analysis environments: Excel, ArcGIS, Matlab, programming languages; MyDB for storage of analysis data

20 HIS Server Supports data discovery, delivery and publication Supports data discovery, delivery and publication Data discovery – how do I find the data I want? Data discovery – how do I find the data I want? Map interface and observations catalogs Map interface and observations catalogs Data delivery – how do I acquire the data I want? Data delivery – how do I acquire the data I want? Use web services or retrieve from local database Use web services or retrieve from local database Data Publication – how do I publish my observation data? Data Publication – how do I publish my observation data? Use Observations Data Model Use Observations Data Model

21 National and Workgroup HIS National HIS has a polygon in it marking the region of coverage of a workgroup HIS server Workgroup HIS has local observations catalogs for coverage of national data sources in its region. These local catalogs are partitioned from the national observations catalogs. For HIS 1.0 the National and Workgroup HIS servers will not be dynamically connected. National HISWorkgroup HIS

22 NWIS ArcGIS Excel NCAR Unidata NASA Storet NCDC Ameriflux Matlab AccessJava Fortran Visual Basic C/C++ Some operational services CUAHSI Web Services Data Sources Applications Extract Transform Load

23 ExcelCUAHSI Web service How Excel connects to ODM Obtains inputs for CUAHSI web methods from relevant cells. Available Web methods are GetSiteInfo, GetVariableInfo GetValues methods. converts standardized request to SQLquery. imports VB object into Excel and graphs it converts response to a standardized XML. Observations Data Model SQL query Response HydroObjects converts XML to VB object parses user inputs into a standardized CUAHSI web method request.

24 Data Types Hydrologic Observation Data Geospatial Data Weather and Climate Data Remote Sensing Data (NetCDF) (GIS) (Relational database) (EOS-HDF) Digital Watershed

25 HIS Extensions Integration of Weather Data Integration of Weather Data Work with NCAR; prototype on Ohio Work with NCAR; prototype on Ohio Move from gridded to watershed-based delivery of data Move from gridded to watershed-based delivery of data Hydrogeology Hydrogeology Constructing stratigraphy for continent Constructing stratigraphy for continent Geologic Framework Geologic Framework Geomorphic and geologic history Geomorphic and geologic history Incorporation of human dimension Incorporation of human dimension Transportation; structures Transportation; structures Permits, Toxic Release Inventory, etc. Permits, Toxic Release Inventory, etc. Flood plain (contribution from real estate sector?) Flood plain (contribution from real estate sector?) FEMA Lidar products FEMA Lidar products Explicit development of AK, HI, PR beyond CONUS Explicit development of AK, HI, PR beyond CONUS

26 Digital Observatories Data Representation Data Representation Need to develop integrated 4-dimensional data base Need to develop integrated 4-dimensional data base Add coverages (easy) and fields (more complicated) Add coverages (easy) and fields (more complicated) Conceptual Frameworks and Modeling Conceptual Frameworks and Modeling Can single data representation fulfill diverse set of science needs? Can single data representation fulfill diverse set of science needs? Data assimilation techniques to guide sensor deployment and operation Data assimilation techniques to guide sensor deployment and operation

27 Environmental Observatories Two Paths: Bottom-up- Critical Zone Observatories (CZO). EAR- Hydrologic Science, Geochemistry/Biogeochemistry, Geomorphology Bottom-up- Critical Zone Observatories (CZO). EAR- Hydrologic Science, Geochemistry/Biogeochemistry, Geomorphology Top-Down- WATERS Network- CLEANER/CUAHSI Top-Down- WATERS Network- CLEANER/CUAHSI

28 Observatory Development: Bottom- up EAR Research funds: CZO Solicitation Two sites selected in 2007 Two sites selected in 2007 CUAHSI will push for public data access CUAHSI will push for public data access If successful, these first two hopefully initial pieces of a prototype observatory network If successful, these first two hopefully initial pieces of a prototype observatory network

29 Observatory Development: Top Down Major Research Equipment and Facility Construction (MREFC) WATERS Network- WATERS Network- Currently 11 testbeds have been either awarded or recommended for award Currently 11 testbeds have been either awarded or recommended for award

An initiative of the U. S. National Science Foundation Engineering and Geosciences Directorates HydroView WATERS Network Presenter Name, Affiliation Presented at …. Date

31 Environmental Observatories: a new approach for integrated, field-oriented collaborative research at regional to continental scales Rely on advances in: sensors and sensor networks at intensively instrumented sites shared by the research community cyberinfrastructure with high bandwidth to connect the sites, data repositories, and researchers into collaboratories distributed modeling platforms From USGS

32 WATERS Network WATer and Environmental Research Systems Network  Joint collaboration between the CLEANER Project Office and CUAHSI, Inc, sponsored by ENG & GEO Directorates at the National Science Foundation (NSF) CLEANER = Collaborative Large Scale Engineering Analysis Network for Environmental Research CLEANER = Collaborative Large Scale Engineering Analysis Network for Environmental Research CUAHSI = Consortium of Universities for the Advancement of Hydrologic Science CUAHSI = Consortium of Universities for the Advancement of Hydrologic Science  Planning underway to build a nationwide environmental observatory network using NSF’s Major Research Equipment and Facility Construction (MREFC) funding

33 The Need…and Why Now? Nothing is more fundamental to life than water. Not only is water a basic need, but adequate safe water underpins the nation’s health, economy, security, and ecology. NRC (2004) Confronting the nation’s water problems: the role of research. ● Water use globally will triple in the next two decades, leading to increases in erosion, pollution, dewatering, and salinization. ● Only ~55% of US river and stream miles and acres of lakes and estuaries fully meet their intended uses; ~45% are polluted, mostly from diffuse-source runoff. ● Polluted runoff caused more than 16,000 beach closings and swimming advisories in the US in 2001 ● Floods cause an average of $5.2 billion/year in damage and 80 deaths/year. ● Of 45,000 U.S. wells tested for pesticides, 5,500 had harmful levels of at least one. ● Fish consumption advisories are common in 30 states because of elevated mercury levels.

34 Four critical deficiencies in current abilities: 1. Basic data about water-related processes at the needed spatial and temporal resolution. 2. The means to integrate data across scales from different media and sources (observations, experiments, simulations). 3. Sufficiently accurate modeling and decision-support tools to predict underlying processes and forecast effects of different engineering management strategies. 4. The understanding of fundamental processes needed to transfer knowledge and predictions across spatial and temporal scales—from the scale of measurements to the scale of a desired management action Courtesy of Tom Harmon

35 DRAFT Vision for the WATERS Network: WATERS is an integrated observing system that will transform our ability to predict the quality and quantity of the nation’s waters in real time.

36 CLEANER GRAND CHALLENGE How do we detect, predict, and manage the effects of human activities and natural perturbations on the quantity, distribution and quality of water in near real time? Develop capacity to detect effects on water of key drivers: Population and land use shifts Energy, water and material resource use Climate change

37 CUAHSI GRAND CHALLENGE Develop a predictive understanding of continental water dynamics and its interaction with climate, landscape, ecology and civilization Three themes in CUAHSI science plan: Patterns and variability Climate and human development Prediction and adaptation to change

38 The WATERS Network will: 1. Consist of: (a) a national network of interacting field sites; across a range of spatial scales, climate and land-use/cover conditions (b) teams of investigators studying human-stressed landscapes, with an emphasis on water problems; (c) specialized support personnel, facilities, and technology; and (d) integrative cyberinfrastructure to provide a shared-use network as the framework for collaborative analysis 2. Transform environmental engineering and hydrologic science research and education 3. Enable more effective management of human-dominated environments based on observation, experimentation, modeling, engineering analysis, and design

39 Common Vision: WATERS Network Sensors and Measurement Facility Synthesis Informatics Observatories/ Environmental Field Facilities

40 Network Design Principles: Enable multi-scale, dynamic predictive modeling for water, sediment, and water quality (flux, flow paths, rates), including: Near-real-time assimilation of data Feedback for observatory design Point- to national-scale prediction Network provides data sets and framework to test: Sufficiency of the data Alternative model conceptualizations Master Design Variables: Scale Climate (arid vs humid) Coastal vs inland Land use, land cover, population density Energy and materials/industry Land form and geology Nested (where appropriate) Observatories over Range of Scales: Point Plot (100 m 2 ) Subcatchment (2 km 2 ) Catchment (10 km 2 ) – single land use Watershed (100–10,000 km 2 ) – mixed use Basin (10,000–100,000 km 2 ) Continental

41 I III II I I Simplified schema of a potential national WATERS Network based on three hydrologic regions: (I) coastal, (II) humid-continental, and (III) arid-continental; large blue circles: regional, watershed-based observatories; small circles: intensively instrumented field sites

42 PrecipitationDeposition Agricultural Fields CAFOs Reservoir Water Treatment Water Treatment Retention Basin Evaporation Humid Continental Watershed: Potential WATERS EFF site Sensor Sensors

43 Sensor Network  Distributed network of interchangeable arrays of remote and embedded stationary and mobile sensors Satellite and Remote Sensing Individual Deployed Sensors Individual Deployed Sensors

44 Potential Observatory WATERS Network Cyberinfrastructure

45 WATERS Network CI Planning  CLEANER Cyberinfrastructure Committee is creating a CI program plan with CUAHSI  Two major projects are creating CI prototypes for WATERS (with more to come): CUAHSI Hydrologic Information System (HIS) project (David Maidment, PI)CUAHSI Hydrologic Information System (HIS) project (David Maidment, PI) NCSA Environmental CI Demonstration (ECID) project (Barbara Minsker and Jim Myers co-leads)NCSA Environmental CI Demonstration (ECID) project (Barbara Minsker and Jim Myers co-leads) Have jointly proposed a draft common environmental CI architecture Have jointly proposed a draft common environmental CI architecture Are demonstrating how observatories can enable adaptive forecasting and management Are demonstrating how observatories can enable adaptive forecasting and management

46 Environmental CI Architecture: Research Services Create Hypo- thesis Obtain Data Analyze Data &/or Assimilate into Model(s) Link &/or Run Analyses &/or Model(s) Discuss Results Publish Knowledge Services Data Services Workflows & Model Services Meta- Workflows Collaboration Services Digital Library Research Process Supporting Technology Integrated CI ECID Project Focus: Cyberenvironments HIS Project Focus

47 Collaboration Services: CyberCollaboratory A web portal to allow sharing of information and ideas across the community. Currently being used by CLEANER Project Office team Public CI demo space shows numerous cyberinfrastructure demonstrations

48 CUAHSI Data Services Web Services Library Web application: Data Portal Your application Excel, ArcGIS, Matlab Fortran, C/C++, Visual Basic Hydrologic model ……………. Your operating system Windows, Unix, Linux, Mac Internet Simple Object Access Protocol Source: David Maidment, Univ. of Texas

49 Path Forward: Timeline 2007 Joint workshop on integrating social sciences into EOs (January) Public comment period on draft program plan and conceptual design (science, sensor, organization, social science, education, and cyberinfrastructure-February) Completion of program plan and conceptual design (August) 2008 Consortium arrangements established Conceptual Design Review - move to MREFC Readiness Status 2009 Continue funding development of enabling technologies, test-beds, and prototypes FY 2012 Start of MREFC funding and construction phase FY 2016 Project completion; start of full-scale operations

50Conclusions  Networked observatories will create a new paradigm for environmental research Shared infrastructure at large scales Shared infrastructure at large scales Interdisciplinary teams collaborating remotely to address complex environmental issues Interdisciplinary teams collaborating remotely to address complex environmental issues Enabling improved understanding & management of large-scale natural environmental systems Enabling improved understanding & management of large-scale natural environmental systems  Cyberinfrastructure will be critical to the observatory’s goals Enabling real-time collaboration, data services, modeling, & management Enabling real-time collaboration, data services, modeling, & management Creating a national knowledge network, not a set of individual field sites and investigators Creating a national knowledge network, not a set of individual field sites and investigators

51 Acknowledgments  100s of researchers and educators from across the US are contributing to WATERS’ planning process  Funding sources: NSF grants BES , BES , EAR , and SCI NSF grants BES , BES , EAR , and SCI Office of Naval Research grant N Office of Naval Research grant N

52 Questions?  Contact info: Jon Duncan, Project Manager How can CUAHSI and CLEANER make better linkages with NWQMC?