1 Dapper and Argo Joe Sirott PMEL/NOAA

2 What is Dapper? Web server that provides distributed access to in-situ data via OPeNDAP protocol Clients include GrADS, Matlab, NcBrowse, Java Ocean Atlas Available data: –PMEL EPIC database –ARGO profiles –NODC World Ocean Database 2001 All components are open-source

3 What does Dapper do? Efficiently aggregates thousands of in-situ data files (> for WODB2001) Handles several in-situ data formats: –Argo GDAC netCDF –PMEL EPIC netCDF –COARDS compliant netCDF Supports time series or profiles Supports space/time queries Based on HPCC funded Climate Data Portal (Soreide, Zhu, Denbo)

4 What is OPeNDAP? OPeNDAP (Open source Project for a Network Data Access Protocol) is a protocol for remotely accessing scientific data Uses HTTP as transport

5 Dapper OPeNDAP Example (space/time query) Client HTTP request: 0&lat<20 Dapper HTTP response: Dataset { Sequence { Sequence { Float32 lat; Float32 lat; Float64 _id; Float64 _id; Sequence { Sequence { Float32 pres,temp; Float32 pres,temp; } profile; } } location = { { -4.01,1, { {4.3,26.808}, {100,24.982}, …}, } location = { { -4.01,1, { {4.3,26.808}, {100,24.982}, …}, { -52, 2, {{8.3,3.77},{100,3.743},…} { -52, 2, {{8.3,3.77},{100,3.743},…}

6 Dapper OPeNDAP Example (single profile) Client HTTP request: Dapper HTTP response: Dataset { Sequence { Sequence { Float64 time; Float64 time; Float32 lat,lon; Float32 lat,lon; Sequence { Sequence { Float32 pres,temp; Float32 pres,temp; } profile; } } location = { { ,-52.9, }, {{8.3,3.77},{9.1,3.77},…}}; } location = { { ,-52.9, }, {{8.3,3.77},{9.1,3.77},…}};

7 Dapper architectural overview Uses relational database to store metadata and coordinate boundaries for each profile/time series file Data maintained in original format All data delivered as OPeNDAP sequences

8 Dapper architecture CORBA (IIOP) Climate Data Portal MySQL Java netCDF library netCDF profile netCDF profile Dapper HTTP Server CDP service netCDF service OPeNDAP client DChart OPeNDAP Protocol (HTTP) Database loader

9 DChart Browser based in-situ OPeNDAP client Based on FY2004 HPCC funded project for shipboard display of cruise data AJAX based user interface

10 DChart Design Goals Highly responsive browser based user interface Low learning curve Rapid previewing and downloading of Dapper datasets Highly interactive map for region selection No Java applets or proprietary plug-ins

11 Dapper architecture diagram CORBA (IIOP) Climate Data Portal MySQL Java netCDF library netCDF profile netCDF profile Dapper HTTP Server CDP service netCDF service OPeNDAP client DChart OPeNDAP Protocol (HTTP) Database loader

12 DChart architecture Web Browser JavaScript DChart ServletDapper OPeNDAP Protocol (HTTP) XML/JavaScript (HTTP) JFreeChart SGT OPeNDAP Client XML Engine

13 DChart Demo NDBC video Argo video NDBC video Argo video NDBC video Argo video

14 CDFsync Rapid synchronization of netCDF in-situ files over a network Only compressed file differences transmitted across network Based on rsync, but optimized for netCDF and file lists consisting of thousands or millions of files Funded by HPCC 2004 (Charles Sun et al) Available at

15 Future plans Constraints (QC flags, attributes) Metadata display Trajectory and section plots Multiple plot windows Save/restore Web app state

16 Questions?

17 More info PMEL Dapper Server PMEL Dapper UI Downloads, propaganda /

18 Back

19 Back