Gridded Data Interoperability for the Integrated Ocean Observing System Rich Signell USGS Woods Hole/NOAA Silver Spring USA USGS CDI Web Meeting: Oct 22,

Slides:



Advertisements
Similar presentations
1 NASA CEOP Status & Demo CEOS WGISS-25 Sanya, China February 27, 2008 Yonsook Enloe.
Advertisements

The Model Output Interoperability Experiment in the Gulf of Maine: A Success Story Made Possible By CF, NcML, NetCDF-Java and THREDDS Rich Signell (USGS,
Spatial Data Infrastructure: Concepts and Components Geog 458: Map Sources and Errors March 6, 2006.
Integrating NOAA’s Unified Access Framework in GEOSS: Making Earth Observation data easier to access and use Matt Austin NOAA Technology Planning and Integration.
NCAR GIS Program : Bridging Gaps
Development of a Community Hydrologic Information System Jeffery S. Horsburgh Utah State University David G. Tarboton Utah State University.
Integrated Ocean Observing System Data Management and Communications March 2004 The US Integrated Ocean Observing System (IOOS) Plan for Data Management.
A Distributed Approach to Ocean, Atmosphere & Climate Model Data Interoperability Rich Signell USGS Woods Hole, MA COAWST Training Workshop Woods Hole,
TPAC Digital Library Talk Overview Presenter:Glenn Hyland Tasmanian Partnership for Advanced Computing & Australian Antarctic Division Outline: TPAC Overview.
A Super-Regional Modeling Testbed for Improving Forecasts of Environmental Processes for the U.S. Atlantic and Gulf of Mexico Coasts Cyberinfrastructure.
Coast Survey Development Laboratory National Ocean Service, NOAA Mary Erickson March 2011.
Coastal and Marine Spatial Planning Debra Hernandez, SECOORA Walter Johnson, BOEMRE Ru Morrison, NERACOOS Charly Alexander, IOOS Office.
A Super-Regional Modeling Testbed for Improving Forecasts of Environmental Processes for the U.S. Atlantic and Gulf of Mexico Coasts Don Wright, SURA Principal.
Implementation of Model Data Interoperability for IOOS: Successes and Lessons Learned Rich Signell USGS Woods Hole, MA / NOAA Silver Spring USA Model Data.
Unidata’s TDS Workshop TDS Overview – Part II October 2012.
2 3 ROMS/COAWST NcML file 4 5 Exploiting IOOS: A Distributed, Standards-Based Framework and Software Stack for Searching, Accessing, Analyzing and.
Unidata TDS Workshop TDS Overview – Part I XX-XX October 2014.
IOOS Data Management Integration Standards Plans in the Northeast Eric Bridger (GMRI) Sep
Gulf of Maine / Scituate Harbor - Extratropical Domain Shelf Hypoxia ChesROMS Long & Hood UMCES Estuarine Hypoxia Inundation Cyber Infrastructure IOOS.
Super-Regional Modeling Testbed to Improve Forecasts of Environmental Processes for the U.S. Atlantic and Gulf of Mexico Coasts Super-Regional Modeling.
A Super-Regional Modeling Testbed for Improving Forecasts of Environmental Processes for the U.S. Atlantic and Gulf of Mexico Coasts Don Wright, SURA Principal.
Mid-Course Review: NetCDF in the Current Proposal Period Russ Rew
Super-Regional Modeling Testbed to Improve Forecasts of Environmental Processes for the U.S. Atlantic and Gulf of Mexico Coasts Wright, L.D.; Signell,
IODE Ocean Data Portal - technological framework of new IODE system Dr. Sergey Belov, et al. Partnership Centre for the IODE Ocean Data Portal MINCyT,
Enhancements to a Community Toolset for Ocean Model Data Interoperability: Unstructured grids, NCTOOLBOX, and Distributed Search Rich Signell (USGS), Woods.
Tools in Support of a National DMAC Derrick Snowden NERACOOS/ODP Annual Meeting 26 Sep 2012.
Accomplishments and Remaining Challenges: THREDDS Data Server and Common Data Model Ethan Davis Unidata Policy Committee Meeting May 2011.
The netCDF-4 data model and format Russ Rew, UCAR Unidata NetCDF Workshop 25 October 2012.
DMAC: Infrastructure to enable delivery of IOOS ® information Presentation to the IOOS Advisory Committee Derrick Snowden, System Architect, U.S. IOOS.
Unidata and Oceanography Through the Ages Rich Signell USGS Coastal and Marine Science Center Woods Hole, MA & NOAA Integrated Ocean Observing System (IOOS)
1 Global Systems Division (GSD) Earth System Research Laboratory (ESRL) NextGen Weather Data Cube Chris MacDermaid October, 2010.
IOOS Model Data Interoperability Design ROMS POM WW3 WRF ECOM NcML Common Data Model OPeNDAP+CF WCS NetCDF Subset THREDDS Data Server Standardized (CF)
Integrated Model Data Management S.Hankin ESMF July ‘04 Integrated data management in the ESMF (ESME) Steve Hankin (NOAA/PMEL & IOOS/DMAC) ESMF Team meeting.
1 NOAA IOOS Program Data Integration Framework (DIF) Project Overview Adapted from a brief to the NOAA Data Management Committee August 6, 2008 by Jeff.
ESIP Federation Air Quality Cluster Partner Agencies.
OOI CI LCA REVIEW August 2010 Ocean Observatories Initiative External Observatory Integration Christopher Mueller, Matt Arrott, John Graybeal Life Cycle.
Building a Web-based GIS Portal For the Great Lakes Observing System (GLOS) Pete Giencke Program Specialist Data and Information Management.
IOOS Modeling Testbed Cyberinfrastructure Rich Signell, USGS, Woods Hole, MA IOOS-RA-Briefing, Feb 14, 2012.
Geosciences - Observations (Bob Wilhelmson) The geosciences in NSF’s world consists of atmospheric science, ocean science, and earth science Many of the.
Draft GEO Framework, Chapter 6 “Architecture” Architecture Subgroup / Group on Earth Observations Presented by Ivan DeLoatch (US) Subgroup Co-Chair Earth.
IOOS Data Services with the THREDDS Data Server Rich Signell USGS, Woods Hole IOOS DMAC Workshop Silver Spring Sep 10, 2013 Rich Signell USGS, Woods Hole.
Unidata’s TDS Workshop TDS Overview – Part I July 2011.
GEON2 and OpenEarth Framework (OEF) Bradley Wallet School of Geology and Geophysics, University of Oklahoma
1 NOAA Priorities for an Ecosystem Approach to Management A Presentation to the NOAA Science Advisory Board John H. Dunnigan NOAA Ecosystem Goal Team Lead.
The Unified Access Framework for Gridded Data … the 1 st year focus of NOAA’s Global Earth Observation Integrated Data Environment (GEO-IDE) Steve Hankin,
The Integrated Ocean Observing System (IOOS) is envisioned as a coordinated national and international network of observations, data management and analyses.
Distributed Data Analysis & Dissemination System (D-DADS ) Special Interest Group on Data Integration June 2000.
Earth System Curator and Model Metadata Discovery and Display for CMIP5 Sylvia Murphy and Cecelia Deluca (NOAA/CIRES) Hannah Wilcox (NCAR/CISL) Metafor.
U.S. INTEGRATED OCEAN OBSERVING SYSTEM DATA ACCESS SERVICES Coastal Geotools: Advances in Federal Data Access Services Hassan Moustahfid, Chris Duncombe.
Vision of an Integrated Global Observing System Gregory W. Withee Assistant Administrator for Satellite and Information Services National Oceanic and Atmospheric.
Convergence And Trust in Earth and Space Science Data Systems Ted Habermann, NOAA National Geophysical Data Center Documentation: It’s not just discovery...
Catalog-driven workflows using CSW Rich Signell, USGS, Woods Hole, MA, USA Filipe Fernandes, SECOORA, Brazil Kyle Wilcox, Axiom Data Science, Wickford,
E u r o p e a n C o m m i s s i o nCommunity Research Global Change and Ecosystems Malta, 27 January 2004 Alan Edwards EUROPEAN COMMISSION GMES – Implications.
National Geospatial Enterprise Architecture N S D I National Spatial Data Infrastructure An Architectural Process Overview Presented by Eliot Christian.
Access to IOOS Data Relevant to OOI Kathleen Bailey NOAA/NOS/IOOS January 6, 2016.
Distributed Data Servers and Web Interface in the Climate Data Portal Willa H. Zhu Joint Institute for the Study of Ocean and Atmosphere University of.
May 2010 GGIM, New York City The National System for Coordination of Territorial Information SNIT NSDI of Chile.
Rich Signell Roland Viger Curtis Price USGS Community for Data Integration Feb 15, 2012.
1 2.5 DISTRIBUTED DATA INTEGRATION WTF-CEOP (WGISS Test Facility for CEOP) May 2007 Yonsook Enloe (NASA/SGT) Chris Lynnes (NASA)
U N I T E D S T A T E S D E P A R T M E N T O F C O M M E R C E N A T I O N A L O C E A N I C A N D A T M O S P H E R I C A D M I N I S T R A T I O N.
Interoperability Day Introduction Standards-based Web Services Interfaces to Existing Atmospheric/Oceanographic Data Systems Ben Domenico Unidata Program.
NOAA IOOS SOS Implementations in 2008 Jeff de La Beaujardière, PhD NOAA IOOS Program DIF Sr Systems Architect.
Update on Unidata Technologies for Data Access Russ Rew
NOAA EDMC Ocean Observatories Initiative Cyberinfrastructure Karen Stocks OOI CI Data Curator University of California, San Diego Ocean Observatories.
Unidata Infrastructure for Data Services Russ Rew GO-ESSP Workshop, LLNL
CEOS Working Group on Information System and Services (WGISS) Data Access Infrastructure and Interoperability Standards Andrew Mitchell - NASA Goddard.
IODE Ocean Data Portal - technological framework of new IODE system Dr. Sergey Belov, et al. Partnership Centre for the IODE Ocean Data Portal.
Associate Director for Research, Education and Marine Operations
US IOOS®, NDBC, CO-OPS Partnership Discussion
WGISS Connected Data Assets Oct 24, 2018 Yonsook Enloe
Presentation transcript:

Gridded Data Interoperability for the Integrated Ocean Observing System Rich Signell USGS Woods Hole/NOAA Silver Spring USA USGS CDI Web Meeting: Oct 22, 2009 Rich Signell USGS Woods Hole/NOAA Silver Spring USA USGS CDI Web Meeting: Oct 22, 2009

US Integrated Ocean Observing System (IOOS ® ) IOOS ® Plan defines: IOOS ® Plan defines: Global ComponentGlobal Component Coastal ComponentCoastal Component  17 Federal Agencies  11 Regional Associations

IOOS Legislation Part of HR146: “Omnibus Public Land Management Act of 2009”, signed by Obama on March 30, 2009 Major step for IOOS, establishing a formal program by law, therefore not vulnerable to possible changing priorities at NOAA Makes the National Oceanic and Atmospheric Administration (NOAA) the System's lead federal agency. Recognizes Regional Associations, and sets forth the requirements to be certified Provides liability coverage to the regions for data dissemination … But does not provide funding

Legislation states that IOOS shall: Support national defense, marine commerce, navigation safety, weather, climate, and marine forecasting, energy siting and production, economic development, ecosystem-based resource management, public safety and public outreach training and education Promote awareness of ocean, coastal, and Great Lakes resources Promote basic and applied scientific research Improve the ability to measure, track, explain, and predict weather and climate change and natural climate variability.

IOOS ® Subsystems Observing Systems Data Management and Communications (DMAC) Modeling and Analysis In-situ & remotely sensed ocean information Decision support tools to aid informed and timely decision making Effectively & Efficiently links ocean information to user applications

DMAC Core Principles Adopt open standards & practices Avoid proprietary protocols or technologies Avoid customer-specific stovepipes Adopt open standards & practices Avoid proprietary protocols or technologies Avoid customer-specific stovepipes 6 Customer Web access service DataProvider Observations Models

Data Management Model Standardized access services implemented at data providers Customers starting to ingest data from these services Standardized access services implemented at data providers Customers starting to ingest data from these services Customer accessservice DataProvider Observations Models

Sensor Systems In situ observations OGC SOS (SensorObservationService) Table of Contents Metadata Data Values DataProvider Metadata QualityControl Observations

Maps of Data OGC WMS (WebMapService) Table of Contents GeoreferencedImages DataProvider satellite HF radar modeloutputs In situ observations

HF radar Gridded Data and Model Outputs Table of Contents Metadata Data Values DataProvider CF Metadata QualityControl satellite griddedmodeloutputs OGC WCS (WebCoverageService) CF+OPeNDAP CF+OPeNDAP

Federated –Several Data Assembly Centers Each supports particular Organization, Region, Theme,... Standardized service interfaces –Shared components Registry, Catalog, Viewer, … Federated –Several Data Assembly Centers Each supports particular Organization, Region, Theme,... Standardized service interfaces –Shared components Registry, Catalog, Viewer, … Federated, Service-Oriented Architecture Monolithic –All data in single warehouse –All services provided by warehouse Fully distributed –Every data provider adopts the same services Catalog Registry Viewer DAC 5 DAC 2 DAC 3 DAC 4 DAC 1 Global DAC CatalogRegistryViewer Midway between these extremes

NOAA IOOS Program Model Data Interoperability for US IOOS (In 12 months or less) Rich Signell USGS Woods Hole/NOAA Silver Spring USA CSIRO Brief: April 6, 2009 Rich Signell USGS Woods Hole/NOAA Silver Spring USA CSIRO Brief: April 6, 2009

The Problem: Stovepiped Model Access

Result: Inability to compare and assess model results & underutilization of model products The GoMOOS Nowcast/Forecast Circulation Model (University of Maine)

The Interoperability Experiment on Model Data in the Gulf of Maine Gulf of Maine Ocean Data Partnership Modeling Committee: Five different groups (UMAINE, UMASS, BIO, WHOI and UNH) Each group had: –A different oceanographic, met or wave model: POM, ECOM, ROMS, WaveWatch 3, WRF –A different toolkit for analysis But each group also had: –NetCDF output files –A public web server (HTTP/1.1) Gulf of Maine Ocean Data Partnership Modeling Committee: Five different groups (UMAINE, UMASS, BIO, WHOI and UNH) Each group had: –A different oceanographic, met or wave model: POM, ECOM, ROMS, WaveWatch 3, WRF –A different toolkit for analysis But each group also had: –NetCDF output files –A public web server (HTTP/1.1)

A Simple NetCDF File

NetCDF Machine independent, self-describing, binary format for multidimensional scientific data Efficient hyperslab extraction of data – nc_varget(file,var,corner,edge,stride) – nc_varget(‘etopo2.nc’,’topo’,[0 0],[20 30],[1 2]) Unidata interfaces: Fortran, C, C++, Java Third party interfaces: Perl, IDL, Python, Matlab MIT-style license, supported by NSF at Unidata Machine independent, self-describing, binary format for multidimensional scientific data Efficient hyperslab extraction of data – nc_varget(file,var,corner,edge,stride) – nc_varget(‘etopo2.nc’,’topo’,[0 0],[20 30],[1 2]) Unidata interfaces: Fortran, C, C++, Java Third party interfaces: Perl, IDL, Python, Matlab MIT-style license, supported by NSF at Unidata

OPeNDAP (f.k.a. DODS) Open-source Project for a Network Data Access Protocol: delivery of multidimensional scientific data via http OPeNDAP allows efficient slicing from data via the web, just as NetCDF works for local files. ETOPO2v2?topo[0:19][0:29] Open-source Project for a Network Data Access Protocol: delivery of multidimensional scientific data via http OPeNDAP allows efficient slicing from data via the web, just as NetCDF works for local files. ETOPO2v2?topo[0:19][0:29]

Issue: Ocean grids are not regularly spaced! Stretched surface and terrain following vertical coordinates Curvilinear orthogonal horizontal coordinates

NetCDF Climate and Forecast (CF) Conventions provide a solution Groups using CF: GO-ESSP: Global Organization for Earth System Science Portal IOOS: Integrated Ocean Observing System ESMF: Earth System Modeling Framework OGC: Open Geospatial Consortium (GALEON: WCS profile)

CF Conventions for horizontal coordinates

CF Conventions for stretched vertical coordinates

CF-Compliant ROMS File

NcML to the Rescue! (XML markup language for NetCDF)

THREDDS Data Server

NcML to the Rescue! (XML markup language for NetCDF)

ROMS POM WW3 WRF ECOM NcML Unidata Common Data Model OPeNDAP OGC WMS NetCDF Subset Service THREDDS Data Server Standardized (CF) & Aggregated Virtual Datasets Nonstandard Output Files Web Services Data Interoperability System Design Matlab Godiva2 Panoply IDV ERDDAP Standard Clients NetCDF -Java API NcML, NetCDF-Java and THREDDS Data Server built and supported by Unidata under NSF-support ArcGIS OGC WCS

NJ Toolkit for Matlab Objective: Make it simple to access CF data Example function: [t, geo]=nj_tslice(URI,’temp’,1); t = 22x120x180 single geo = –lat: [120x180 single] –lon: [120x180 single] – z: [22x120x180 double] –time: (matlab datenum) nj_tslice works identically for ROMS, POM, ECOM, WRF, Wavewatch3 URI can be: local NetCDF, remote NetCDF, NcML, OpenDAP Data URL Mostly by Sachin Kumar Bhate MSSTATE Objective: Make it simple to access CF data Example function: [t, geo]=nj_tslice(URI,’temp’,1); t = 22x120x180 single geo = –lat: [120x180 single] –lon: [120x180 single] – z: [22x120x180 double] –time: (matlab datenum) nj_tslice works identically for ROMS, POM, ECOM, WRF, Wavewatch3 URI can be: local NetCDF, remote NetCDF, NcML, OpenDAP Data URL Mostly by Sachin Kumar Bhate MSSTATE

GOMODP THREDDS Catalog

Web Mapping Service with ncWMS

3D visualization of data with IDV

Comparing Models with Data in Matlab Model 1: UMASS-ECOM Model 2: UMAINE-POM Data: SST 2008-Sep-08 07:32

Comparing Models with Data in Matlab

Jim Manning’s drifter comparison

Buoy_comp.m

Web Mapping Service with ncWMS

Working with Data in ArcGIS using the NOAA/ASA Environmental Data Connector

Extract to multiple formats using ERDDAP

Extracting Bathymetry Data: Step1: Browse for OpenDAP URL

Extracting Bathy Data: Step 2: Use cf_subsetGrid.m with OpenDAP URL

bathy_comp.m

Using Mirone for Bathymetry

Exported from Mirone to Iview3D

Summary What did we get through the use of this approach? –Standards-based delivery of aggregated model data with minimal burden on the data provider –Single point of access for regional model results –Model data interoperability via standard software –Multiple modes of access for users –More eyes on the model results –More usage of model results –Faster feedback to modelers –Improved models! We now have an IOOS THREDDS Catalog that points to 17 THREDDS servers running in all 11 IOOS Regions This approach can be applied to any gridded data What did we get through the use of this approach? –Standards-based delivery of aggregated model data with minimal burden on the data provider –Single point of access for regional model results –Model data interoperability via standard software –Multiple modes of access for users –More eyes on the model results –More usage of model results –Faster feedback to modelers –Improved models! We now have an IOOS THREDDS Catalog that points to 17 THREDDS servers running in all 11 IOOS Regions This approach can be applied to any gridded data

Ocean Model Native Grid Type #2 (triangular unstructured grid)

Immediate Ocean Modeling Infrastructure Needs Develop support for unstructured grids: Need a new Scientific Feature type in the CDM, new class and methods in NetCDF-Java, delivery via WMS in TDS, and visualization in IDV. Develop protocols and back end software in the TDS to allow server side functions such as subsetting and averaging functions. More standardized tool development (e.g. virtual instrument suite) Strengthen ties between Unidata, the Integrated Ocean Observing System (IOOS), and the NSF-sponsored Ocean Observing Initiative CyberInfrastructure (OOI- CI) project. Develop support for unstructured grids: Need a new Scientific Feature type in the CDM, new class and methods in NetCDF-Java, delivery via WMS in TDS, and visualization in IDV. Develop protocols and back end software in the TDS to allow server side functions such as subsetting and averaging functions. More standardized tool development (e.g. virtual instrument suite) Strengthen ties between Unidata, the Integrated Ocean Observing System (IOOS), and the NSF-sponsored Ocean Observing Initiative CyberInfrastructure (OOI- CI) project.

Questions, Feedback? Dr. Richard P. Signell +1 (508) Try the NJ Toolkit for Matlab:

Glossary (1/6) CF Conventions: NetCDF Climate and Forecast Metadata Conventions (Standards for identifying geospatial coordinates in NetCDF files and grid relationships) DMAC: Data Management And Communications subsystem (One of three subsystems of IOOS, providing the infrastructure that will link the Observing and the Modeling and Applications subsystems) ECOM: Estuarine, Coastal, and Ocean Model. (3D ocean circulation model developed principally by Alan Blumberg) ERDDAP: Environmental Research Division's Data Access Program (A web service that aggregates data from diverse remote sources and offers a simple, consistent way to access the data) CF Conventions: NetCDF Climate and Forecast Metadata Conventions (Standards for identifying geospatial coordinates in NetCDF files and grid relationships) DMAC: Data Management And Communications subsystem (One of three subsystems of IOOS, providing the infrastructure that will link the Observing and the Modeling and Applications subsystems) ECOM: Estuarine, Coastal, and Ocean Model. (3D ocean circulation model developed principally by Alan Blumberg) ERDDAP: Environmental Research Division's Data Access Program (A web service that aggregates data from diverse remote sources and offers a simple, consistent way to access the data)

Glossary (2/6) ESMF: Earth System Modeling Framework (Software for building and coupling weather, climate, and related models) GO-ESSP: Global Organization for Earth System Science Portal (Collaboration designed to develop a new generation of software infrastructure that will provide distributed access to observed and simulated data from the climate and weather communities) IDV: Integrated Data Viewer (Java-based software for analyzing and visualizing geoscience data) IOOS: Integrated Ocean Observing System (Multidisciplinary system designed to collect, deliver, and use ocean information) IWGOO: Interagency Working Group on Ocean Observations ESMF: Earth System Modeling Framework (Software for building and coupling weather, climate, and related models) GO-ESSP: Global Organization for Earth System Science Portal (Collaboration designed to develop a new generation of software infrastructure that will provide distributed access to observed and simulated data from the climate and weather communities) IDV: Integrated Data Viewer (Java-based software for analyzing and visualizing geoscience data) IOOS: Integrated Ocean Observing System (Multidisciplinary system designed to collect, deliver, and use ocean information) IWGOO: Interagency Working Group on Ocean Observations

Glossary (3/6) MAST: Modeling and Analysis Steering Team (Guide the development of ocean and coastal modeling activities in support of the goals of IOOS) NcWMS: Web Map Service (ncWMS is a Web Map Service for geospatial data that are stored in CF-compliant NetCDF files. ) NcML: NetCDF Markup Language (Used to standardize and aggregate non-standard NetCDF files) NetCDF-Java: NetCDF-Java Library (Library that takes advantage of CF conventions, NcML to facilitate building standards-based clients) MAST: Modeling and Analysis Steering Team (Guide the development of ocean and coastal modeling activities in support of the goals of IOOS) NcWMS: Web Map Service (ncWMS is a Web Map Service for geospatial data that are stored in CF-compliant NetCDF files. ) NcML: NetCDF Markup Language (Used to standardize and aggregate non-standard NetCDF files) NetCDF-Java: NetCDF-Java Library (Library that takes advantage of CF conventions, NcML to facilitate building standards-based clients)

Glossary (4/6) NSF: National Science Foundation (independent federal agency created by Congress in 1950 "to promote the progress of science; to advance the national health, prosperity, and welfare; to secure the national defense…“) OpenDap: Open-source Project for a Network Data Access Protocol (allows efficient hyperslabbing of geospatial data) POM: Princeton Ocean Model (Sigma coordinate, free surface ocean model with embedded turbulence and wave sub-models, and wet-dry capability) RCOOS: Regional Coastal Ocean Observing System (Generic term for one of the 11 regional observing systems in IOOS) NSF: National Science Foundation (independent federal agency created by Congress in 1950 "to promote the progress of science; to advance the national health, prosperity, and welfare; to secure the national defense…“) OpenDap: Open-source Project for a Network Data Access Protocol (allows efficient hyperslabbing of geospatial data) POM: Princeton Ocean Model (Sigma coordinate, free surface ocean model with embedded turbulence and wave sub-models, and wet-dry capability) RCOOS: Regional Coastal Ocean Observing System (Generic term for one of the 11 regional observing systems in IOOS)

Glossary (5/6) ROMS: Regional Ocean Modeling System (A free-surface, terrain-following, primitive equations ocean model widely used by the scientific community for a diverse range of applications) SCCOOS: Southern California Coastal Ocean Observing System (Brings together coastal observations in the Southern California Bight to provide information necessary to address issues in climate change, ecosystem preservation and management, coastal water quality, maritime operations, coastal hazards and national security. One of the 11 IOOS regions.) THREDDS (Thematic Realtime Environmental Distributed Data Services) (Services for delivering standards-based geospatial data via OpenDAP, WCS and more, works with NcML) ROMS: Regional Ocean Modeling System (A free-surface, terrain-following, primitive equations ocean model widely used by the scientific community for a diverse range of applications) SCCOOS: Southern California Coastal Ocean Observing System (Brings together coastal observations in the Southern California Bight to provide information necessary to address issues in climate change, ecosystem preservation and management, coastal water quality, maritime operations, coastal hazards and national security. One of the 11 IOOS regions.) THREDDS (Thematic Realtime Environmental Distributed Data Services) (Services for delivering standards-based geospatial data via OpenDAP, WCS and more, works with NcML)

Glossary (6/6) WCS: Web Coverage Service (OGC standard for serving gridded information: time series, profiles, hyperslabs of remote sensing data, bathymetry, model output) WW3: WaveWatch III (Ocean wave model developed by Hendrik Tolman) WRF: Weather Research and Forecasting Model (Next-generation mesocale numerical weather prediction system designed to serve both operational forecasting and atmospheric research needs) WCS: Web Coverage Service (OGC standard for serving gridded information: time series, profiles, hyperslabs of remote sensing data, bathymetry, model output) WW3: WaveWatch III (Ocean wave model developed by Hendrik Tolman) WRF: Weather Research and Forecasting Model (Next-generation mesocale numerical weather prediction system designed to serve both operational forecasting and atmospheric research needs)

Path from research to operations Recommendation: - R01 funding, to support (a) Education & Training and (b) Research Grade modeling (broadly defined) activities – open to all academia - R02 funding, to support semi-operational systems at regional associations - R03 funding to support operational systems at federal agencies - R04 funding, to support translation and evaluation

Towards an integrated modeling system for the U.S. coastal margin … … with clearly delineated roles, responsibilities and sustaining mechanisms == Draft document == Prepared by A. Baptista with input from D. Schwab and many others, for discussion by the NFRA modeling sub-committee Version 1.0, March 2009 == Draft document == Prepared by A. Baptista with input from D. Schwab and many others, for discussion by the NFRA modeling sub-committee Version 1.0, March 2009

Background: Hot off the press IOOS legislation (1) establish a national integrated System of ocean, coastal, and Great Lakes observing systems, comprised of Federal and non-Federal components coordinated at the national level by the National Ocean Research Leadership Council and at the regional level by a network of regional information coordination entities, and that includes in situ, remote, and other coastal and ocean observation, technologies, and data management and communication systems, … (2) improve the Nation's capability to measure, track, explain, and predict events related directly and indirectly to weather and climate change, natural climate variability, and interactions between the oceanic and atmospheric environments, including the Great Lakes; and (3) authorize activities to promote basic and applied research to develop, test, and deploy innovations and improvements in coastal and ocean observation technologies, modeling systems, and other scientific and technological capabilities to improve our conceptual understanding of weather and climate, ocean-atmosphere dynamics, global climate change, physical, chemical, and biological dynamics of the ocean, coastal and Great Lakes environments, and to conserve healthy and restore degraded coastal ecosystems. (1) establish a national integrated System of ocean, coastal, and Great Lakes observing systems, comprised of Federal and non-Federal components coordinated at the national level by the National Ocean Research Leadership Council and at the regional level by a network of regional information coordination entities, and that includes in situ, remote, and other coastal and ocean observation, technologies, and data management and communication systems, … (2) improve the Nation's capability to measure, track, explain, and predict events related directly and indirectly to weather and climate change, natural climate variability, and interactions between the oceanic and atmospheric environments, including the Great Lakes; and (3) authorize activities to promote basic and applied research to develop, test, and deploy innovations and improvements in coastal and ocean observation technologies, modeling systems, and other scientific and technological capabilities to improve our conceptual understanding of weather and climate, ocean-atmosphere dynamics, global climate change, physical, chemical, and biological dynamics of the ocean, coastal and Great Lakes environments, and to conserve healthy and restore degraded coastal ecosystems.

Towards an integrated modeling system for the U.S. coastal margin … … with clearly delineated roles, responsibilities and sustaining mechanisms == Draft document == Prepared by A. Baptista with input from D. Schwab and many others, for discussion by the NFRA modeling sub-committee Version 1.0, March 2009 == Draft document == Prepared by A. Baptista with input from D. Schwab and many others, for discussion by the NFRA modeling sub-committee Version 1.0, March 2009

Background: Hot off the press IOOS legislation (1) establish a national integrated System of ocean, coastal, and Great Lakes observing systems, comprised of Federal and non-Federal components coordinated at the national level by the National Ocean Research Leadership Council and at the regional level by a network of regional information coordination entities, and that includes in situ, remote, and other coastal and ocean observation, technologies, and data management and communication systems, … (2) improve the Nation's capability to measure, track, explain, and predict events related directly and indirectly to weather and climate change, natural climate variability, and interactions between the oceanic and atmospheric environments, including the Great Lakes; and (3) authorize activities to promote basic and applied research to develop, test, and deploy innovations and improvements in coastal and ocean observation technologies, modeling systems, and other scientific and technological capabilities to improve our conceptual understanding of weather and climate, ocean-atmosphere dynamics, global climate change, physical, chemical, and biological dynamics of the ocean, coastal and Great Lakes environments, and to conserve healthy and restore degraded coastal ecosystems. (1) establish a national integrated System of ocean, coastal, and Great Lakes observing systems, comprised of Federal and non-Federal components coordinated at the national level by the National Ocean Research Leadership Council and at the regional level by a network of regional information coordination entities, and that includes in situ, remote, and other coastal and ocean observation, technologies, and data management and communication systems, … (2) improve the Nation's capability to measure, track, explain, and predict events related directly and indirectly to weather and climate change, natural climate variability, and interactions between the oceanic and atmospheric environments, including the Great Lakes; and (3) authorize activities to promote basic and applied research to develop, test, and deploy innovations and improvements in coastal and ocean observation technologies, modeling systems, and other scientific and technological capabilities to improve our conceptual understanding of weather and climate, ocean-atmosphere dynamics, global climate change, physical, chemical, and biological dynamics of the ocean, coastal and Great Lakes environments, and to conserve healthy and restore degraded coastal ecosystems.

Background: Hot off the press IOOS legislation (1) establish a national integrated System of ocean, coastal, and Great Lakes observing systems, comprised of Federal and non-Federal components coordinated at the national level by the National Ocean Research Leadership Council and at the regional level by a network of regional information coordination entities, and that includes in situ, remote, and other coastal and ocean observation, technologies, and data management and communication systems, … (2) improve the Nation's capability to measure, track, explain, and predict events related directly and indirectly to weather and climate change, natural climate variability, and interactions between the oceanic and atmospheric environments, including the Great Lakes; and (3) authorize activities to promote basic and applied research to develop, test, and deploy innovations and improvements in coastal and ocean observation technologies, modeling systems, and other scientific and technological capabilities to improve our conceptual understanding of weather and climate, ocean-atmosphere dynamics, global climate change, physical, chemical, and biological dynamics of the ocean, coastal and Great Lakes environments, and to conserve healthy and restore degraded coastal ecosystems. (1) establish a national integrated System of ocean, coastal, and Great Lakes observing systems, comprised of Federal and non-Federal components coordinated at the national level by the National Ocean Research Leadership Council and at the regional level by a network of regional information coordination entities, and that includes in situ, remote, and other coastal and ocean observation, technologies, and data management and communication systems, … (2) improve the Nation's capability to measure, track, explain, and predict events related directly and indirectly to weather and climate change, natural climate variability, and interactions between the oceanic and atmospheric environments, including the Great Lakes; and (3) authorize activities to promote basic and applied research to develop, test, and deploy innovations and improvements in coastal and ocean observation technologies, modeling systems, and other scientific and technological capabilities to improve our conceptual understanding of weather and climate, ocean-atmosphere dynamics, global climate change, physical, chemical, and biological dynamics of the ocean, coastal and Great Lakes environments, and to conserve healthy and restore degraded coastal ecosystems.

Anecdotal background: modeling post-doc search Stale No response Courtesy response Suggested alternative contact/list Suggested candidate Applied Findings/implications: There is a significant dearth of expertise: most responses knew of no candidates most of suggested candidates were not own students (several duplicates) a significant number or responses asked for access to any good “leftover list” There is a need for investment in education and training of modelers Based on 71 direct s

Goal Establish the need for a sustainable, integrated U.S. coastal margin modeling system as key technology within IOOS Establish the framework for such a system … –… with clearly defined roles and responsibilities for the federal government and for academia … –… in a partnership effectively facilitated through IOOS and its Regional Associations Establish the need for a sustainable, integrated U.S. coastal margin modeling system as key technology within IOOS Establish the framework for such a system … –… with clearly defined roles and responsibilities for the federal government and for academia … –… in a partnership effectively facilitated through IOOS and its Regional Associations

Who does ? DAILY FORECASTS OperationalSemi-operationalResearch grade BasinsNavyN/A ShelvesNOAARA partnershipsRAs, RA partnerships, academia at large Major estuaries/bay s NOAARAsRAs, academia at large Other estuaries/bay s --RAsRAs, academia at large

Semi-operational Path from research to operations Research grade Operational Education and training Translation

Moving across operational levels Research grade Operation al Semi- operation al ? ? ?? Need mechanism to make evaluation/review objective, and to facilitate evolution of modeling technology and strategy.

Open Columbia River benchmark Goals Enablers Phases Concept Code developers Community CMOP Joint Color Key Goals  Enable continuous enhancement of multiple models and exploration of diverse modeling strategies  Maximize value-added expertise of model developers/expert users, while minimizing their time investment Illustration only. Think of any (or selected?) IOOS site of interest

Path from research to operations Recommendation: - R01 funding, to support (a) Education & Training and (b) Research Grade modeling (broadly defined) activities – open to all academia - R02 funding, to support semi-operational systems at regional associations - R03 funding to support operational systems at federal agencies - R04 funding, to support translation and evaluation

Definitions Operational –Runs reliably and robustly, at prescribed frequency –Distributes consistent products at scheduled times –Is regularly quality controlled –Has 24/7 maintenance Semi-operational –Runs reliably and robustly, at prescribed frequency –Distributes consistent products at scheduled times –Is regularly quality controlled Research grade –May run only an ad hoc basis –Has no mandate for product distribution –May be quality controlled, but perhaps only on an exploratory basis –May be bottom-up –… Operational –Runs reliably and robustly, at prescribed frequency –Distributes consistent products at scheduled times –Is regularly quality controlled –Has 24/7 maintenance Semi-operational –Runs reliably and robustly, at prescribed frequency –Distributes consistent products at scheduled times –Is regularly quality controlled Research grade –May run only an ad hoc basis –Has no mandate for product distribution –May be quality controlled, but perhaps only on an exploratory basis –May be bottom-up –…

Goal Establish the need for a sustainable, integrated U.S. coastal margin modeling system as key technology within IOOS Establish the framework for such a system … –… with clearly defined roles and responsibilities for the federal government and for academia … –… in a partnership effectively facilitated through IOOS and its Regional Associations Establish the need for a sustainable, integrated U.S. coastal margin modeling system as key technology within IOOS Establish the framework for such a system … –… with clearly defined roles and responsibilities for the federal government and for academia … –… in a partnership effectively facilitated through IOOS and its Regional Associations

Basic principles / assumptions Operational systems need a long-term institutional framework –The federal government does this best No operational system will be sustainable without an adequate workforce –Academia does this best No operational system will be effective without the benefits of regional knowledge and regular innovation –Academia does this best A healthy operational system requires an effective and sustained “translation” layer between academia and the federal government –IOOS, specifically through Regional Associations, has the opportunity to do this best –Note: “Translation” needs definition. It includes more than one- way export of software and data. It is a funded exchange, of experiences, technologies, and evaluation methods Operational systems need a long-term institutional framework –The federal government does this best No operational system will be sustainable without an adequate workforce –Academia does this best No operational system will be effective without the benefits of regional knowledge and regular innovation –Academia does this best A healthy operational system requires an effective and sustained “translation” layer between academia and the federal government –IOOS, specifically through Regional Associations, has the opportunity to do this best –Note: “Translation” needs definition. It includes more than one- way export of software and data. It is a funded exchange, of experiences, technologies, and evaluation methods