Crossing the Digital Divide CUAHSI Crossing the Digital Divide Presented by: Fernando R. Salas David Maidment, Enrico Boldrini, Ben Domenico, Stefano Nativi CUAHSI Conference on Hydrologic Data and Information Systems Logan, Utah April 22, 2011

Introduction and Motivation Source: Iowa State University Department of Natural Resource Ecology and Management

Searching for Climate Data

What is the Digital Divide? Observations and Time Series Climate Observations and Grids Remote Sensing Geographic Information Systems (GIS)

Interoperability Challenges Observations Data Gridded Data Measures variables over a discrete space-time domain Stored in tables, relational databases and geodatabases Measures variables over a continuous space-time domain Stored in large multi-dimensional arrays in specialized formats How can we connect these very different worlds together?

Hydrologic Information Systems

Observations Data Information Model Data Source United States Geological Survey (USGS) Network National Water Information System – Unit Values Sites Colorado River at Austin Variables Streamflow Values 2,500 cfs at 8:30 am on June 19th, 2011 Conceptually, we describe hydrologic observations data with a Point Observations Information Model.

Gridded Data Information Model Data Source National Climatic Data Center (NCDC) Dataset North American Regional Reanalysis (NARR) Grid Size 32 kilometer Variables Temperature Values 78 deg. Fahrenheit at 8:00 am on June 15th, 2000 Time 1 NARR

Data Cube Time {t} Time {t} Space {x,y} Space {x,y,z} Variable {vi}

THREDDS Data Servers WCS NetCDF HTTP OpenDAP WMS

Distributed File Sharing Systems Enables users to access large quantities of gridded data using standardized data services Source: http://www.esrl.noaa.gov/thredds/catalog/Datasets/NARR/catalog.html

Building a Series Catalog for Observations Data Datacart – a collection of data series organized thematically Observations Metadata Catalog Source Site Name Location (Lat/Long) Variable Name Variable Units Concept Start Date End Date WaterML URI WFS

Building a Series Catalog for Gridded Data Datacart – a collection of data series organized thematically Observations Metadata Catalog Source Coverage Name Location (Lat/Long Box) Variable Name Variable Units Concept Start Date End Date WCS URI WFS

Building a Catalog of Services CSW Time Series Raster Series WFS WFS CSW implemented all over! HydroServers WaterML NetCDF THREDDS Servers

Services Stack Framework Catalog Metadata Data Services CSW WFS SOS WCS Catalog Services for the Web (CSW) enable metadata services describing observations data to be registered in a catalog of services. Web Feature Services (WFS) enable data services to be completely described by metadata in a series catalog. Sensor Observation Services (SOS) and Web Coverage Services (WCS) enable observations and climate data to be conveyed through the internet in a standardized manner.

Services Stack Framework for Observations Data Catalog Metadata Data Services CSW WFS SOS Catalog Services for the Web (CSW) enable metadata services describing observations data to be registered in a catalog of services. Web Feature Services (WFS) enable observations data services to be completely described by metadata in a series catalog. Sensor Observation Services (SOS) enable observations data to conveyed through the internet in a standardized manner.

Services Stack Framework for Gridded Data Catalog Metadata Data Services CSW WFS WCS Catalog Services for the Web (CSW) enable metadata services describing gridded data to be registered in a catalog of services. Web Feature Services (WFS) enable gridded data services to be completely described by metadata in a series catalog. Web Coverage Services (WCS) enable gridded data to conveyed through the internet in a standardized manner.

One Common Ontology CUAHSI Ontology Precipitation Precip Rainfall Rain Streamflow Turbidity Snow Cloud Cover Precip Rainfall Rain Total Precip Precip Rainfall Rain Total Precip CUAHSI Ontology

HydroPortal – A Catalog Service Geoportal Extension of ArcGIS implements Catalog Services for the Web (CSW) …enables data discovery across multiple data sources and types…

GI-cat Service Broker GI-cat broker catalog mediation/adaptation solution Standard based solution Support to several international and community standards, including THREDDS, WCS, WFS, CSW. Support to several standard discovery interfaces, including CSW 2.0.2 AP ISO, OpenSearch Successful harvesting of more than 400,000 resources Courtesy of Enrico Boldrini from the Earth and Space Science Informatics Lab

A SOA Broker/Mediator component Service Registry Publish  Find  Service Consumer Client Harvest Service Provider  Bind Order  Bind  Server Protocol sharing Service Broker Courtesy of Enrico Boldrini from the Earth and Space Science Informatics Lab Mediator

GI-cat Broker/Mediation component Service Providers Registry Processing capability Service Consumer Client OpenModeller WPS Local Repository /Cache INSPIRE portal Harvest Discovery capability CSW-Core CSW ISO GeoNetwork CSW client GI-cat Service Broker / Mediator OpenSearch CSW ebRIM CSW/ebRIM-EO GBIF ArcGIS Explorer CSW client CSW-ISO Inventory capability Functions: Mediation Distribution Messaging Persistence Access ..... THREDDS/OPeNDAP GEO portal CDI WCS WMS WFS ESA-HMA portal CSW-EBRIM CIM/EO Access capability WCS WFS WMS Web browser THREDDS/OPeNDAP GBIF OpenSearch Mediator Courtesy of Enrico Boldrini from the Earth and Space Science Informatics Lab Server

Interoperability Enabled by Standard Web Services Data Services Catalog Services HydroServers CSW CSW THREDDS Servers HydroCatalog

Integrated Search

Hydrometeorologic Information System HydroCatalog HydroDesktop HydroServer WFS CSW SOS WCS

Conclusions Data access through web services can mask the variations in data structure between relational databases and data file systems

Questions?