Download presentation
Presentation is loading. Please wait.
Published byJessie Walters Modified over 9 years ago
1
Integrating netCDF and OPeNDAP (The DrNO Project) Dr. Dennis Heimbigner Unidata Go-ESSP Workshop Seattle, WA, Sept 17-19 2008
2
Overview Primary goal: Integrate client-side DAP protocol into netCDF C Library Access any DAP data source (thru DAP server) using the netCDF API Initially netCDF-3, later netCDF-4 Rationale: Combine two commonly used API’s for access to scientific datasets Issues: Data model translation DAP dataset URL support (Server side: transparent access to netCDF-4 data)
3
DAP Data Model Primitive types: byte, (u)int16, (u)int32, float32, float64, string Arrays: FORTRAN style rectangular arrays with bounded dimensions Limited naming of dimensions Structure: heterogeneous collection of fields Analog to C/C++ Structs Sequence: variable length array of Structures Allows relational constraints
4
DAP Data Model (cont.) Grid: Combination of an n-dimensional array with n 1-dimensional mapping arrays In effect a structure for an array plus its coordinate variables (in netCDF-speak) Structures, Grids, and Sequences may be arbitrarily nested with each other All types are “singletons” Type reuse requires repeating the definition
5
Specifying a DAP Data Source A DAP data source is specified using an extended URL syntax that refers to the DAP server containing that data Format: ? & Client parameters: [name=value]… URL extension specific to the DAP/netCDF integration Base URL: e.g. http://.../...http://.../ Points to the DAP server
6
Specifying a DAP source (cont.) DAP URL also specifies constraints on the data to be returned by the server Projection: variable-name[first:stride:last] Returns a slice of a rectangular array Selection: boolean expression over variables E.g. x > 5 or y < 6 Only applies to sequences
7
netCDF-3 (aka classic) Data Model Primitive types: char, byte, short, int, float, double Named shared dimensions N-dimensional FORTRAN style arrays Single unlimited dimension May only occur as first (slowest changing) dimension E.g. int var(unlimited,lat,long)
8
netCDF-3 Translation Issues Result must conform to legal classic model E.g. no nested sequences or arrays of sequences Synthesize shared dimensions Infer from DAP dimension name and value Convert grids to equivalent netCDF-3 coordinate variable convention Coordinate variable = 1-d variable with same name as a dimension Contains coordinate values for that dimension Flatten non-dimensioned structures and grids Sequence = unlimited dimension 1-d array
9
netCDF-4 (aka enhanced) Data Model Derived from the HDF5 data model netCDF-3 model plus: More primitives: ubyte, ushort, uint (u)int64, string, enums, opaque (fixed length byte strings) Named user defined types: Compound (=Structure) Vlen – variable length 1-d array Arbitrary use of unlimited dimensions Groups: similar to file system directory tree Group can contain types, dimensions, and variables
10
netCDF-4 Translation Issues netCDF-4 is effectively a superset of the current OPenDAP data model Carryover issues from netCDF-3: Inference of shared dimensions Grid translation to coordinate variable convention Translate structures, grids, and sequences to compound types or maybe groups? Explore DAP data model extensions to include selected netCDF Enhanced concepts Esp. groups and shared dimensions
11
Server-side issues Desirable to be able to pass a netCDF-4 file through a DAP server to a DAP client and through the translation and get the same file Information is currently lost in translation Solutions: add various attribute tags to restore missing information Extend OPenDAP data model
12
Status netCDF 4.1-alpha: available now Libdap+libnc-dap version integrated into current netCDF snapshot build Supports translation of subset of the DAP protocol to netCDF-3 Requires C++ netCDF 4.1-beta: end of 2008 Utilizes Ocapi + modified netCDF => no C++ Limited translation similar to libnc-dap netCDF 4.1: 2009 Utilizes Ocapi + modified netCDF Complete support for translating DAP to netCDF-4 Java version also exists now Uses somewhat different translation rules
13
Acknowledgement NSF Award #0721628 Title: SDCI NMI Improvement: OPeNDAP and NetCDF Integration Principal Investigators: James Gallagher (opendap.org) and Russell Rew (Unidata)
Similar presentations
© 2025 SlidePlayer.com. Inc.
All rights reserved.