Peter Cornillon University of Rhode Island Presented at the 10 September 2003 NVODS Workshop Washington DC The National Virtual Ocean Data System (NVODS): A Status Report

Workshop Objectives  Inform the community – NVODS status  Solicit input with regard to the future direction of NVODS.  Encourage participation in the NVODS effort.

Workshop Objectives 1.Develop a Comprehensive IOOS Data Model 2.Deliver time-critical (real-time) data to data assembly and operational modeling sites 3.Develop DMAC Middleware 4.Make data available using IOOS middleware solution 5.Data Manipulation Services 6.Develop Metrics and Implement Performance Monitoring 7.Implement Middleware Security (Cross-discipline effort with all DMAC) 8.Provide guaranteed geo-temporal-referenced browse for all IOOS data

To Inform - Outline  Overview of NVODS  OPeNDAP – the NVODS core infrastructure  NVODS/OPeNDAP Status  The future

What is NVODS?  NVODS is operated from URI, but involves significant participation from a number of oceanographic institutions in the US and abroad.  NVODS involves for oceanographic data all data system aspects from data discovery through delivery to the user’s application.  Data transport within NVODS is based on the OPeNDAP data access protocol.

Specifically, NVODS  Facilitates access to oceanographic data and data products via the Internet regardless of:  data type,  where the data are stored,  the format in which they are stored,  the user's visualization package, or  the user's level of expertise,  Is envisioned as a component of IOOS, the Integrated Ocean Observing System,

NVODS  Has been desgined to evolve with  the Internet,  computer hardware,  computer software, and  the interests of its users,  Involves a major population effort, and  Makes use of the GCMD, the Global Change Master Directory, for data discovery.

NVODS does NOT deal with  The acquisition of data at the sensor,  Issues related to the long term storage of these data,  A system processing capability,  The actual network (hardware) infrastructure, nor  The very low level data transport protocols, such as tcp/ip.

NVODS and the End-to-End Data System

NVODS Evolved from DODS DODS, the Distributed Oceanographic Data System begun in the early 1990s, consisted of two fundamental parts:  a discipline independent core infrastructure for moving data on the net,  a discipline specific portion related to data – population, location, specialized clients, etc.

DODS  NVODS & OPeNDAP To isolate the discipline independent part of the system from the discipline specific part, two entities have been formed:  National Virtual Ocean Data System (NVODS)  Open Source Project for a Network Data Access Protocol (OPeNDAP)

OPeNDAP  OPeNDAP was formed to maintain and evolve the DODS core infrastructure,  OPeNDAP is a non-profit corporation, and  OPeNDAP is a discipline neutral data access protocol.

OPeNDAP in the End-to-End Data System

NVODS/OPeNDAP Status

OPeNDAP/NVODS Server Sites

Institutions with OPeNDAP Servers  Carolinas Coastal Ocean Observing and Prediction System (Caro-COOPS)  Center for Ocean Land Atmosphere Studies (COLA)  Columbia University/LDEO - International Research Institute (IRI/LDEO)  Florida State University - Center for Ocean-Atmospheric Prediction Studies (COAPS)  George Mason University - Seasonal to Interannual Earth Science Information Partner  Gulf of Maine Ocean Observing System (GoMOOS)  Maine - Department of Marine Resources (Maine DMR)  Monterey Bay Aquarium Research Institute (MBARI)  Naval Oceanographic Office (NAVOCEANO)  NASA/GSFC - Goddard Distributed Active Archive Center (GDAAC)  NASA/JPL - Physical Oceanography Distributed Active Archive Center (PODAAC)  NASA/JPL - Ocean Earth Science Information Partner (OceanESIP)  NCAR - Vegetation/Ecosystem Modeling and Analysis Project (VEMAP)  New York University - Center of Atmosphere Ocean Science  NOAA - Climate Diagnostic Center (CDC)  NOAA - Geophysical Fluid Dynamics Laboratory (GFDL)  NOAA - National Center for Environmental Prediction (NCEP)  Universities/NGO Research Labs  State Organizations  NOAA  NASA  USGS  Navy

Institutions with OPeNDAP Servers  NOAA - National Climatic Data Center (NCDC)  NOAA - National Data Buoy Center (NDBC)  NOAA - Northeast Fisheries Science Center (NMFS/NEFSC)  NOAA - Pacific Marine Environmental Laboratory (PMEL)  Oregon State University - Remote Sensing Ocean Optics (ORSOO)  Science Applications International Corporation (SAIC)  Texas A&M University (TAMU)  UCAR - Unidata Internet Data Distribution (IDD)  United States Geological Survey - Woods Hole Field Center (USGS)  University of Alabama/Huntsville - Global Hydrology Resource Center (GHRC)  University of Hawaii/SOEST - International Pacific Research Center  University of Hawaii/SOEST - Sea Level Center  University of Kansas - Kansas Geological Survey  University of Massachusetts - School of Marine Science and Technology  Universite Pierre et Marie Curie - Laboratoire de Meteorlogie Dynamique du C.N.R.S  University of Rhode Island - Graduate School of Oceanography  Woods Hole Oceanographic Institution - US/GLOBEC Georges Bank Program (GLOBEC)  Woods Hole Oceanographic Institution - Upper Ocean Mooring Data Archive (UOP)  Universities/NGO Research Labs  State Organizations  NOAA  NASA  USGS  Navy

OPeNDAP Client and Server Status

Special Servers

Interesting OPeNDAP Access Statistics LDEO data accesses for 1 st quarter of 2002 TypeRequests%Volume (gb)% OPeNDAP191, Other2,062, Total2,254, TypeRequests%Volume (gb)% OPeNDAP643, Other3,264, Total3,908, , LDEO data accesses for 4 th quarter of 2002

The Future  Near term plans – existing funding (sort of)  New funding  Challenges

Near Term Plans  OPeNDAP  DAP 4  Provide web service access to the DAP  XML version of the DAP  SOAP interface to the DAP  Add bit-image data type  Add cataloging capabilities to OPeNDAP-servers using THREDDS technology  Extend Ancillary Information Services (AIS) capabilites  Complete the OPeNDAP data Connector

Near Term Plans  NVODS  Populate Ancillary Information Services (AIS) site for oceanographic data  Work with groups implementing pilots

New Funding  NASA REASoN CAN – Thematic Portals for Satellite-Derived Surface Ocean Parameters  Between $500 and $750/year to NVODS  About ½ of funding for OPeNDAP  Will provide OPeNDAP/LAS access to satellite-derived SST, Surface Winds, Ocean Color, Precipitation and SSH data sets.  NOPP – GODAE  $70k/year to OPeNDAP  Will work with the data assimilation community to make their data accessible via OPeNDAP/LAS

Challenges  Data discovery – Directory population  GIS access  Consistent semantic data model(s)

Challenges  Data discovery – Directory population  Finding new servers and new data sets at existing servers  OPeNDAP-enabled servers are rapidly being installed  New data sets are being added at existing server sites  Obtaining complete and consistent data set descriptions  Inadequate data set descriptions  No link between the data set description and OPeNDAP access to the data.  Data sets consisting of collections of holdings pose special problems  Each of the individual items may be well defined, but not the collection as a whole, or  There may not be a single entry point for the collection

Challenges  Data discovery – Directory population (cont.)  Handling virtual data sets and server services  New data sets obtained by combining data from several sites The collection of mooring data obtained from several sites  New data objects obtained via operations on the data at a site Wind stress curl derived from the wind stress field  Handling replicated data sets  Some of the more popular data sets are replicated at several sites

Challenges  GIS access  Access from GIS-based servers  Grid access  Feature access  Access from GIS packages  Grid access  Feature access

Challenges  Consistent semantic data model(s)

NVODS: NVODS: OPeNDAP: OPeNDAP: DODS: DODS: Data System URLS