May 2003National Coastal Data Development Center Brief Introduction Two components Data Exchange Infrastructure (DEI) Spatial Data Model (SDM) Together, they make up “NCDDC Middleware”

May 2003National Coastal Data Development Center Brief Introduction to NCDDC Middleware - what it does and why Richard Owens

May 2003National Coastal Data Development Center Brief What Is DEI (High-level overview) DEI is a client/server framework by which applications and data repositories can interact. The goal of the framework is to solve the problems of Data Access and Data Representation. DEI is a CORBA-based set of interfaces which are used to locate and transfer geospatial data. It is a client/server architecture which uses CORBA's IIOP (uses TCP) as its transport protocol.

May 2003National Coastal Data Development Center Brief DEI Concepts Gateway Server component that adapts DEI interfaces to a specific data source Data source dependent DEI Client Any software that uses DEI CORBA interfaces to communicate with gateways ClientGateway Data CORBA

May 2003National Coastal Data Development Center Brief DEI Concepts (continued) Data Flavor (formats) Like a MIME type

May 2003National Coastal Data Development Center Brief (graphic here) What DEI Is Not Complete Web services Lacks generic query processing Poor support for very large datasets Data Archive/Storage Data Format Supports them, though

May 2003National Coastal Data Development Center Brief History Originally Navy NRL funded project Now being developed by NCDDC Current Status Java based gateways via Java GDK Current focus C++ gateways Original focus, now lagging behind Java Future Mature Java GDK, then C++ GDK Add generic query processing Add support for very large datasets Open source

May 2003National Coastal Data Development Center Brief What is the SDM Object-oriented library that is used to represent and manipulate spatial data objects. 3D Some other dimensional capabilities, too Structured Grids Regular, Rectilinear, Curvilinear, Projected Unstructured Grids Coordinate Systems Cartographic, Cartesian, Cylindrical, Spherical GIS-like structures Points, Lines, Polygons, etc.

May 2003National Coastal Data Development Center Brief What is the SDM (continued) Java ~160 classes ~1050 Unit tests

May 2003National Coastal Data Development Center Brief Key Features Called the “Field Spatial Data Model” by its developer because it uses a ‘field’ concept, which relates spatial objects to data. Fundamental storage is multi-dimensional array. Decoupling of data storage from its geometry and topology. Requires basic metadata (quantity, units, validity, accuracy, etc.) for all data. Aids in interoperability

May 2003National Coastal Data Development Center Brief Small, consistent API for data access. The SDM hides all implementation details of data representation behind its abstract interfaces. Decouples many issues regarding storage and internal representation of the data, while providing a consistent interface to it. Data may be logically represented one way and physically stored another way. Many manipulations may be done without actually having to have the data physically in memory. Key Features (continued)

May 2003National Coastal Data Development Center Brief Externalized format for representing datasets XML based (FDML) Can be generated without the use of the SDM. Key Features (continued)

May 2003National Coastal Data Development Center Brief Field Concept ARRAY MESHFIELD DATA ARRAY DATA

May 2003National Coastal Data Development Center Brief What SDM Is Not Transport mechanism Storage/Archival Magic. Won’t solve all problems. Just a data format Complete

May 2003National Coastal Data Development Center Brief History EPA project Ported from Eiffel to Java in 2002

May 2003National Coastal Data Development Center Brief Future Documentation FY03 Text data (done) Generalize dimensional support beyond existing 3-D (to 2-D, 1-D or N-D) Investigate Time dimension support (complete in FY04) Application Layer Services Common Query mechanism Subsetting (parameters, spatial) Format support (NetCDF, Shapefile) FY04 Aggregation of data types Time dimensions support - including subsetting in Application Layer Open Source

May 2003National Coastal Data Development Center Brief NCDDC Middleware Objective Develop a middleware architecture that can be used to tie data and applications together across a local or wide-area network in a way that is independent of the data and application platforms, operating systems, and formats.

May 2003National Coastal Data Development Center Brief NCDDC Middleware Defined “NCDDC Middleware” is the fusion of DEI with SDM DEI Data Transport SDM Data Representation DEI + SDM (“NCDDC Middleware”) Spatial data representation distributed over a network Common form for data, and the means for transporting it

May 2003National Coastal Data Development Center Brief DEI/SDM, HTTP/HTML Analogy Transport HTTP, DEI Content HTML, SDM

May 2003National Coastal Data Development Center Brief Benefits Gateways can be created that can transport descriptions of the data with embedded links to the actual data – the actual transport of the data is done piecewise, “ on demand ”.  Can have datasets that are larger than memory.  Caching mechanism improves performance. Data is transported in binary form  Sun’s XDR format Foundation for netCDF  Fast and efficient Since DEI is integrated within the SDM, clients can use SDM interfaces to access gateways without the need to code to DEI ’ s CORBA interfaces.