Dispatch Layer and the NetCDF Architecture

Slides:

Advertisements

Similar presentations

A Draft Standard for the CF Metadata Conventions Cheryl Craig and Russ Rew UCAR.

Advertisements

A Common Data Model In the Middle Tier Enabling Data Access in Workflows … HDF/HDF-EOS Workshop XIV September 29, 2010 Doug Lindholm Laboratory for Atmospheric.

Streaming NetCDF John Caron July What does NetCDF do for you? Data Storage: machine-, OS-, compiler-independent Standard API (Application Programming.

THREDDS, CDM, OPeNDAP, netCDF and Related Conventions John Caron Unidata/UCAR Sep 2007.

7 +/- 2 Maybe Good Ideas John Caron June (1) NetCDF-Java (aka CDM) has lots of functionality, but only available in Java – NcML Aggregation – Access.

HDF5 OPeNDAP Project Update and Demo MuQun Yang and Hyo-Kyung Lee (The HDF Group) James Gallagher (OPeNDAP, Inc.)

iRODS: Interoperability in Data Management

Christine White, Esri Growing OPeNDAP Support: Current ArcGIS Workflows and Future Directions Christine White, Esri

NetCDF Ed Hartnett Unidata/UCAR

Unidata TDS Workshop THREDDS Data Server Overview October 2014.

Overview of the ODP Data Provider Sergey Sukhonosov National Oceanographic Data Centre, Russia Expert training on the Ocean Data Portal technology, Buenos.

Toward Rich, User- Defined Aggregation & Subset-Selection Services Dave Fulker, President, OPeNDAP, Inc ESIP Summer Meeting on 9-12 July 2013 Thursday.

OPeNDAP and the Data Access Protocol (DAP) Original version by Dave Fulker.

The HDF Group ESIP Summer Meeting HDF OPeNDAP update Kent Yang The HDF Group 1 July 8 – 11, 2014.

NetCDF for Developers and Data Providers Russ Rew, UCAR Unidata ICTP Advanced School on High Performance and Grid Computing 14 April 2011.

Unidata’s TDS Workshop TDS Overview – Part II October 2012.

Unidata TDS Workshop TDS Overview – Part I XX-XX October 2014.

Mid-Course Review: NetCDF in the Current Proposal Period Russ Rew

Accomplishments and Remaining Challenges: THREDDS Data Server and Common Data Model Ethan Davis Unidata Policy Committee Meeting May 2011.

The HDF Group ESIP Summer Meeting HDF Studio John Readey The HDF Group 1 July 8 – 11, 2014.

Ocean Observatories Initiative Data Management (DM) Subsystem Overview Michael Meisinger September 29, 2009.

Remote programs and commands In this presentation… –rpc concepts –rpc connections –rpc actions.

HDF5 OPeNDAP Project Update and Demo MuQun Yang and Hyo-Kyung Lee (The HDF Group) James Gallagher (OPeNDAP, Inc.) 1HDF and HDF-EOS Workshop XII10/17/2008.

Integrating netCDF and OPeNDAP (The DrNO Project) Dr. Dennis Heimbigner Unidata Go-ESSP Workshop Seattle, WA, Sept

DAP4 James Gallagher & Ethan Davis OPeNDAP and Unidata.

Unidata TDS Workshop THREDDS Data Server Overview

Accessing Remote Datasets using the DAP protocol through the netCDF interface. Dr. Dennis Heimbigner Unidata netCDF Workshop August 3-4, 2009.

Advanced Utilities Extending ncgen to support the netCDF-4 Data Model Dr. Dennis Heimbigner Unidata netCDF Workshop August 3-4, 2009.

Unidata’s TDS Workshop TDS Overview – Part I July 2011.

HDF4 OPeNDAP Project Progress Report MuQun Yang and Hyo-Kyung Lee 1 HDF Developers' Meeting11/24/2015.

Remote Data Access with OPeNDAP Dr. Dennis Heimbigner Unidata netCDF Workshop October 25, 2012.

HDF5 OPeNDAP Project Update and Demo MuQun Yang and Hyo-Kyung Lee (The HDF Group) James Gallagher (OPeNDAP, Inc.) 1 HDF and HDF-EOS Workshop XII10/17/2008.

HDF5 OPeNDAP Project Update and Demo MuQun Yang and Hyo-Kyung Lee (The HDF Group) James Gallagher (OPeNDAP, Inc.) 1HDF and HDF-EOS Workshop XII, Aurora,

The HDF Group Data Interoperability The HDF Group Staff Sep , 2010HDF/HDF-EOS Workshop XIV1.

The HDF Group Introduction to netCDF-4 Elena Pourmal The HDF Group 110/17/2015.

Department of Computer Science Southern Illinois University Edwardsville Spring, 2008 Dr. Hiroshi Fujinoki FTP Protocol Programming.

The NetCDF/OPeNDAP (NO) Project or “What part of NO do you not understand?”

11/8/2007HDF and HDF-EOS Workshop XI, Landover, MD1 Software to access HDF5 Datasets via OPeNDAP MuQun Yang, Hyo-Kyung Lee The HDF Group.

LAS and THREDDS: Partners for Education Roland Schweitzer Steve Hankin Jonathan Callahan Joe Mclean Kevin O’Brien Ansley Manke Yonghua Wei.

A Draft Standard for the CF Metadata Conventions Russ Rew, Unidata GO-ESSP 2009 Workshop

Reiner Schlitzer Alfred Wegener Institute for Polar and Marine Research ODV – New Developments.

9/21/04 James Gallagher Server-Side: The Basics This part of the workshop contains an overview of the two servers which OPeNDAP has developed. One uses.

Update on Unidata Technologies for Data Access Russ Rew

Utilities for netCDF-4 Dr. Dennis Heimbigner Unidata Advanced netCDF Workshop July 25, 2011.

TSDS (HPDE DAP). Objectives (1) develop a standard API for time series-like data, (2) develop a software package, TSDS (Time Series Data Server), that.

Core and Framework DIRAC Workshop October Marseille.

ISC321 Database Systems I Chapter 2: Overview of Database Languages and Architectures Fall 2015 Dr. Abdullah Almutairi.

HDF5 OPeNDAP Project Update and Demo MuQun Yang and Hyo-Kyung Lee (The HDF Group) James Gallagher (OPeNDAP, Inc.) 1HDF and HDF-EOS Workshop XII, Aurora,

Object Interaction: RMI and RPC 1. Overview 2 Distributed applications programming - distributed objects model - RMI, invocation semantics - RPC Products.

Product Designer Hub – Taking HPD to the Web

DAP+NETCDF Using the netCDF-4 Data Model

Chapter 5 Remote Procedure Call

Plans for an Enhanced NetCDF-4 Interface to HDF5 Data

netCDF Support for In-Memory Files

Efficiently serving HDF5 via OPeNDAP

Remote Data Access Update

Extending the NetCDF Supported Data Formats using a Dispatch Layer

MATLAB, netCDF, and OPeNDAP

DATABASE LINK DISTRIBUTED DATABASE.

Unidata Advanced netCDF Workshop

Access HDF5 Datasets via OPeNDAP’s Data Access Protocol (DAP)

CdmCL, a Specific Textual Constraint Language for Common Data Model

Remote Data Access Update

ROMS/TOMS Web Sites & ROMS: NetCDF and Matlab

Accessing Remote Datasets through the netCDF interface.

Chapter 15: File System Internals

NCL variable based on a netCDF variable model

OPeNDAP Community Working Groups October 18, 2007

Future Development Plans

OPeNDAP/Hyrax Interfaces

Presentation transcript:

Dispatch Layer and the NetCDF Architecture Dr. Dennis Heimbigner Unidata netCDF Workshop July 26, 2011

Overview Goal: netCDF as universal API Dispatch Architecture Dispatch table selection rules Questions

Goal: netCDF as the universal API Actual semantics of nc_open or nc_create determined by choice of dispatch table Currently support: netCDF-3 (classic), netCDF-4, OPeNDAP, and CDM-Remote Proposed additions: HDF-4 EOS, DAP4, RPC Allow for user-community defined extensions

Dispatch Architecture Path Mode flags Path Prefix Existing File Client Program nc_open() nc_get_var() … nc_close() netCDF API Wrapper Procedures nc_open() nc_get_var() … nc_close Dispatch Table For e.g. Netcdf-4

Dispatch Selection Rules The dispatch table is chosen in the NC_create and the NC_open wrapper procedures The decision is based on the following pieces of information. Path – regular file, url, or something else Path prefix – e.g. [cdmremote]<url> Mode argument – a specific flag (e.g. NC_NETCDF4) Contents of existing file (when using nc_open) NC_set_dispatch_override() Environment variables (not currently used)

Questions?