Recent developments and trends in Network Access to Oceanographic In-situ Data Nancy Soreide, NOAA/PMEL John “Oz” Osborne, NOAA/PMEL - OceanAtlas Software Joe Sirott, NOAA/PMEL – Sirott and Associates Tiffany Vance – NOAA/AFSC Nazila Merati – NOAA/PMEL – JISAO Christopher Moore – NOAA/PMEL - JISAO American Meteorological Society Meeting January 29-Feb 3, 2006, Atlanta, GA

What is OPeNDAP?  OPeNDAP (Open source Project for a Network Data Access Protocol) is a protocol for remotely accessing scientific data  Uses HTTP as transport  Endorsed as a standard by IOOS Data Management and Communications (DMAC) committee Joe Sirott

But…  OPeNDAP in-situ data is poor stepchild of OPeNDAP gridded data  Few desktop clients support it  Even fewer Web apps support it  Some recent progress (GrADS, Java OceanAtlas) but change is slow Joe Sirott

What is Dapper?  Web server that provides distributed access to in-situ data via OPeNDAP protocol  Clients include GrADS, Matlab, LAS, NcBrowse, Java Ocean Atlas, DChart  Available data:  Realtime global Argo profiles  NDBC Time Series  WOCE Sea Level  PMEL in-situ data (EPIC database)  Coming: TAO & NODC World Ocean Database 2001  All components are open-source Joe Sirott

What does Dapper do?  Efficiently aggregates thousands of in-situ data files (> for WODB2001)  Handles several in-situ data formats:  Argo GDAC netCDF  PMEL EPIC netCDF  COARDS compliant netCDF  Supports time series or profiles  Supports space/time queries  Based on HPCC funded Climate Data Portal (Soreide, Zhu, Denbo) Joe Sirott

Dapper architectural overview  Uses relational database to store metadata and coordinate boundaries for each profile/time series file  Data maintained in original format  All data delivered as OPeNDAP sequences Joe Sirott

Dapper architecture diagram CORBA (IIOP) Climate Data Portal (CDP) MySQL Java netCDF library netCDF profile netCDF profile Dapper HTTP Server CDP service netCDF service OPeNDAP clients OPeNDAP Protocol (HTTP) Database loader Joe Sirott

DChart  Browser based in-situ OPeNDAP client  Based on FY2004 HPCC funded project for shipboard display of cruise data  AJAX based user interface  AJAX is used by Google Maps Joe Sirott

DChart Design Goals  Highly responsive browser based user interface  Low learning curve  Rapid previewing and downloading of Dapper datasets  Highly interactive map for region selection  No Java applets or proprietary plug-ins Joe Sirott

Ocean Data Navigator (aka DCHART) New! Joe Sirott

Ocean Data Navigator (aka DChart) New! Joe Sirott

AJAX The Good, the Bad and the Ugly Joe Sirott

Good stuff  Much more responsive user interface  Desktop application functionality in familiar Web browser interface  Client responsible for state management  Zero effort software updates (for users!)  “Cool” factor Joe Sirott

Bad stuff  Requires “modern” browser  DChart support currently limited to IE 6, Firefox 1.0.x, Mozilla  Covers > 90% of users and alternative would be downloading desktop app  Debugging is difficult  A lot for developers to learn:  XML,XPath,XSLT,JavaScript, CSS, DHTML, DOM…  Libraries/frameworks immature  Maintenance and testing difficult Joe Sirott

And the ugly…  Browser incompatibilities still an issue:  GET limited to 2048 characters in IE  IE window coordinate system different than Firefox  IE memory leaks  Legacy ActiveX reference counting  Transparent images with alpha channel  Disappearing DOM nodes with XSLT in IE  Ugly kludges required:  GET/POST through Iframe to retain history Joe Sirott

Lessons Learned  Desktop applications are moving to the browser.  It’s easier to move data to servers than servers to data.  The response time of the back end is critical in AJAX applications.  It takes a lot more time to develop the back end than the user interface. Joe Sirott

Java OceanAtlas Comprehensive Profile Data Analysis for the Desktop Data types: –Supports many profile data formats (bottle and CTD data) –Argo –Distributed data (OPeNDAP/dapper) –Gridded netCDF data –Time series Graphics: –Property-property, profile, contour, “station” value, and map plots –Overlay section contour plots –Contour plots on maps –Color plots by any parameter or metadata (date/time, longitude, or latitude) –Color map symbols by station calculation values –Presentation quality graphics and choice of fonts Calculations: –Large selection of standard oceanographic calculations –Station calculations: mixed-later depth, interpolation to surface, neutral surface, extrema, statistics –Section calculations: section difference and mean cast John Osborne

Java OceanAtlas 10°N Residuals O2: WOCE vs. World Ocean Atlas Surface O2 John Osborne

OceanGIS Integration of GIS and Java tools  Read simple shape files  Read ocean data from dapper  Read model output  Rotate/pan/zoom  Hyperlink objects and 3D “picking”  Switch from 2D projections to 3D  GIS tools  ArcObjects, ArcGIS, ArcIMS, ArcGIS Engine  Java tools  MapObjects Java, Intra-Layer Calculator, VTK (Java wrapper), Java 3D, Oceanographic tools (Java), GeoTools Tiffany Vance, Christopher Moore, Nazila Merati

Accessing ocean profile sensor data from database (selected profiles in blue) OceanGIS Dapper Data Access Tiffany Vance, Christopher Moore, Nazila Merati

CTD cast location (light blue) Mixed-layer depth (dark blue) Tiffany Vance, Christopher Moore, Nazila Merati

Resulting ocean profile sections analyzed with GIS statistical analysis tools Tiffany Vance, Christopher Moore, Nazila Merati

Summary  Dapper – OPeNDAP server for ocean observations data  DChart – AJAX Web client for dapper  Java OceanAtlas – Desktop java client for dapper  OceanGIS – uses dapper for access to ocean profiles  Other dapper clients – ncBrowse, GrADS, LAS More information: