OPeNDAP/Hyrax Interfaces James Gallagher

What are...? OPeNDAP: A not-for-profit corporation which steers the growth of the DAP. All software it develops is open source. DAP: Data Access Protocol - a simple request-response protocol which uses HTTP to move structured (scientific) data from a server to a client. Hyrax: A data server developed by OPeNDAP that supports DAP.

Types of Interfaces Application Server Software libraries Component

Application Level DAP Requests made with a URL DAP Responses returned as MIME documents DAP is stateless; implementation using REST DAP response form is a ‘universal representation’ (canonical) for data - it’s generated by a server (nominally) on the fly without any need to modify the original data. DAP servers for over a dozen very different formats exist DAP Servers also support ‘Web’ features Other protocols that work with the Web (e.g., THREDDS) easily coexist with DAP See Data Access Protocol (DAP), version 2

Server: Hyrax Front end (OLFS); Java; Processes requests and uses the BES Back end (BES); C++; Builds responses and responds to OLFS commands N-Tier architecture OLFS and BES are extensible See Hyrax

Hyrax: Many Geometries Simplest... one host or two OLFS BES OLFS BES One OLFS talks to many BES daemons BES OLFS WCS Many OLFS front-ends talk to one BES

Components: BES Format handlers: Read data Response handlers: Build new responses Reporters: Record usage information Aggregators: Build aggregate responses Implement C++ classes New modules are loaded at runtime See Hyrax: For Developers, Extending the BES

Components: OLFS Extending the OLFS New Dispatch handlers See Writing Custom DIspatch Handlers

Software Libraries From OPeNDAP: C++, C, Java OPeNDAP/Unidata: NetCDF Other groups: Python

OPeNDAP, DAP and Hyrax OPeNDAP: Runs the open-source project developing and maintaining DAP. OPeNDAP makes a conscious effort to work with other groups. DAP: A simple protocol, able to take advantage of HTTP’s wide-spread use, that has proven to work well with a variety of data. Hyrax: A modular, extensible, multi- protocol server for DAP.

More Detail about DAP…

DAP Request Processing DAP requests are made using URLs URL: <pathname>.<request>[?<constraint>] Pathname: names a unique data source Request: what the client wants Constraint: optional modification to the request

Request Processing The OLFS parses the URL into the pathname, request and constraint It uses its knowledge of the BES commands & capabilities to instruct the BES to build the response that will satisfy the request It packages the resulting response from the BES (e.g., builds a HTTP ‘text/plain’ response document and inserts the correct content).

DAP Constraints DAP characterizes data sources as a collection of variables (name-type-value tuples). Constraints are used to ‘project’ and ‘select’ particular variables with a data source. Using a constraint, part of one variable may be accessed/read/returned from a data source with many variables. NB: DAP also uses ‘attributes’ to characterize data sources, but attributes don’t play a direct role in the access of/to binary data values.

BES Handler Interaction Handlers can: Build responses directly from data Use output from other handlers Handlers respond to commands from the ‘command handler’ The command handler can be replaced to replace the BES’s default command syntax Handlers are not limited to DAP

DAP Objects Metadata Syntax - essentially variable declarations that describe the names and types of the variables in the data source. Referred to as DDS - Dataset Descriptor Structure. Use - Information about those variables. Often read from the ‘attributes’ in a file (e.g., HDF4) or from some other source. Referred to as DAS - Dataset Attribute Structure. Data (referred to as ‘DataDDS’) Combines DDS for the requested variables along with binary data values. Binary data are encoded in XDR.

DAP Responses DAP responses are returned by HTTP as ‘documents’ Metadata - both DAS and DDS use text/plain Data - uses application/octet-stream Alternatives: Content of both metadata documents may be returned as a single XML response (text/xml) and Data as a multipart document

Building Responses Responses to requests are built from the one or more of the basic objects. Handlers in the BES ‘pass around’ DAP objects, using them as a generalized representation of the data source. Handlers in the BES that generate responses can effectively be combined with any ‘format’ handler.’

Other Responses Data may be processed and it’s return form changed (e.g., ‘return as ASCII’) Metadata may be used to build ‘info pages’ or Web Uis Other information may be returned (software & protocol version numbers, data source organization, et c.)